# Tests with coverage in VSCode

**URL:** <https://discourse.julialang.org/t/tests-with-coverage-in-vscode/117787>\
**Category:** General Usage\
**Tags:** testing, vscode\
**Created:** [August 3, 2024, 4:11pm UTC](https://discourse.julialang.org/t/tests-with-coverage-in-vscode/117787 "2024-08-03T16:11:35Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![brianguenter](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/brianguenter/32/29519_2.png) [@brianguenter](https://discourse.julialang.org/u/brianguenter)\
**Post date:** [August 3, 2024, 4:11pm UTC](https://discourse.julialang.org/t/tests-with-coverage-in-vscode/117787/1 "2024-08-03T16:11:35Z")

</div>

I recently noticed for the first time this icon in the Julia extension for testing in VSCode:

 ![image](https://global.discourse-cdn.com/julialang/original/3X/e/c/ec54026efece728281a11e788089e787381bafb0.png)

The hover tool tip says “run tests with coverage”. I ran it and got a message saying this feature only works in Julia v 1.11. Installed 1.11 and ran it again - it generated coverage information for a few files then hung.

For the few files with coverage information it was great - I could see the coverage data in the VSCode editor without having to do any manual processing of coverage files.

Will this feature work when 1.11 releases? It would be super handy and convenient.

---

<div class="post-metadata">

**Author:** ![aplavin](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/aplavin/32/222056_2.png) [@aplavin](https://discourse.julialang.org/u/aplavin)\
**Post date:** [August 3, 2024, 4:52pm UTC](https://discourse.julialang.org/t/tests-with-coverage-in-vscode/117787/2 "2024-08-03T16:52:26Z")

</div>

Note that there’s “Run Test task” → “Julia: run tests with coverage”. It has been available for years, works for any Julia version, and also displays the results directly in VS Code.

Not sure what the new feature does, but I think it only works on Julia 1.12 and that version won’t be officially released for many months from now – even 1.11 is still not released.

---

<div class="post-metadata">

**Author:** ![gdalle](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/gdalle/32/27854_2.png) [@gdalle](https://discourse.julialang.org/u/gdalle)\
**Post date:** [August 3, 2024, 6:02pm UTC](https://discourse.julialang.org/t/tests-with-coverage-in-vscode/117787/3 "2024-08-03T18:02:12Z")

</div>

> [@aplavin](#):
>
> Note that there’s “Run Test task” → “Julia: run tests with coverage”. It has been available for years, works for any Julia version, and also displays the results directly in VS Code.

I wasn’t aware of that! Is it a VSCode feature or a Julia extension feature? Any links to a doc?

---

<div class="post-metadata">

**Author:** ![brianguenter](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/brianguenter/32/29519_2.png) [@brianguenter](https://discourse.julialang.org/u/brianguenter)\
**Post date:** [August 3, 2024, 10:46pm UTC](https://discourse.julialang.org/t/tests-with-coverage-in-vscode/117787/4 "2024-08-03T22:46:28Z")

</div>

Tried “Run Test Task…” on my FastDifferentiation package. It didn’t work for me. Maybe some configuration needs to be set up. I looked at the VSCode docs on this subject but they are long and complicated. Didn’t see how to fix it.

Got this error:

```julia
ERROR: LoadError: The following package names could not be resolved:
 * FastDifferentiation.jl (not found in project or manifest)
   Suggestions:
Stacktrace:
 [1] pkgerror(msg::String)
   @ Pkg.Types C:\Users\seatt\.julia\juliaup\julia-1.11.0-rc2+0.x64.w64.mingw32\share\julia\stdlib\v1.11\Pkg\src\Types.jl:71
 [2] ensure_resolved(ctx::Pkg.Types.Context, manifest::Pkg.Types.Manifest, pkgs::Vector{Pkg.Types.PackageSpec}; registry::Bool)
   @ Pkg.Types C:\Users\seatt\.julia\juliaup\julia-1.11.0-rc2+0.x64.w64.mingw32\share\julia\stdlib\v1.11\Pkg\src\Types.jl:1043
 [3] ensure_resolved
   @ C:\Users\seatt\.julia\juliaup\julia-1.11.0-rc2+0.x64.w64.mingw32\share\julia\stdlib\v1.11\Pkg\src\Types.jl:994 [inlined]
 [4] test(ctx::Pkg.Types.Context, pkgs::Vector{Pkg.Types.PackageSpec}; coverage::Bool, test_fn::Nothing, julia_args::Cmd, test_args::Cmd, force_latest_compatible_version::Bool, allow_earlier_backwards_compatible_versions::Bool, allow_reresolve::Bool, kwargs::@Kwargs{io::IOContext{IO}})
   @ Pkg.API C:\Users\seatt\.julia\juliaup\julia-1.11.0-rc2+0.x64.w64.mingw32\share\julia\stdlib\v1.11\Pkg\src\API.jl:473
 [5] test(pkgs::Vector{Pkg.Types.PackageSpec}; io::IOContext{IO}, kwargs::@Kwargs{coverage::Bool})
   @ Pkg.API C:\Users\seatt\.julia\juliaup\julia-1.11.0-rc2+0.x64.w64.mingw32\share\julia\stdlib\v1.11\Pkg\src\API.jl:159
 [6] test
   @ C:\Users\seatt\.julia\juliaup\julia-1.11.0-rc2+0.x64.w64.mingw32\share\julia\stdlib\v1.11\Pkg\src\API.jl:147 [inlined]
 [7] #test#74
   @ C:\Users\seatt\.julia\juliaup\julia-1.11.0-rc2+0.x64.w64.mingw32\share\julia\stdlib\v1.11\Pkg\src\API.jl:146 [inlined]
 [8] top-level scope
   @ c:\Users\seatt\.vscode\extensions\julialang.language-julia-1.105.2\scripts\tasks\task_test.jl:8
in expression starting at c:\Users\seatt\.vscode\extensions\julialang.language-julia-1.105.2\scripts\tasks\task_test.jl:8

 * The terminal process "C:\Users\seatt\AppData\Local\Microsoft\WindowsApps\julia.exe '--color=yes', '--project=c:\Users\seatt\Source\FastDifferentiation.jl', 'c:\Users\seatt\.vscode\extensions\julialang.language-julia-1.105.2\scripts\tasks\task_test.jl', 'c:\Users\seatt\Source\FastDifferentiation.jl', 'true'" terminated with exit code: 1. 
 * Terminal will be reused by tasks, press any key to close it. 

```

---

<div class="post-metadata">

**Author:** ![aplavin](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/aplavin/32/222056_2.png) [@aplavin](https://discourse.julialang.org/u/aplavin)\
**Post date:** [August 4, 2024, 10:15am UTC](https://discourse.julialang.org/t/tests-with-coverage-in-vscode/117787/5 "2024-08-04T10:15:48Z")

</div>

I think it’s sensitive to the folder name: should be `FastDifferentiation`, not `FastDifferentiation.jl`.

---

<div class="post-metadata">

**Author:** ![brianguenter](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/brianguenter/32/29519_2.png) [@brianguenter](https://discourse.julialang.org/u/brianguenter)\
**Post date:** [August 4, 2024, 6:40pm UTC](https://discourse.julialang.org/t/tests-with-coverage-in-vscode/117787/6 "2024-08-04T18:40:07Z")

</div>

This is the command line that is executed by `Task:Run Test Task` on the `FastDifferentiation.jl` project:

```julia
julia.exe '--color=yes', '--project=c:\Users\seatt\source\FastDifferentiation.jl', 'c:\Users\seatt\.vscode\extensions\julialang.language-julia-1.105.2\scripts\tasks\task_test.jl', 'c:\Users\seatt\source\FastDifferentiation.jl', 'true'

```

and this does not execute properly.

I looked at the `task-test.jl` file and the problem is clear there. Here are the relevant lines:

```julia
path_to_test = ARGS[1]
delete_cov_files = ARGS[2]

package_name_to_run = basename(path_to_test)

using Pkg

Pkg.test(package_name_to_run, coverage=true)

```

If you run `basename` on the path to `FastDifferentiation.jl` you get this:

```julia
julia> basename("c:/Users/seatt/source/FastDifferentiation.jl")
"FastDifferentiation.jl"

```

when what you need is the package name `FastDifferentiation`.

If your github project is named “\*.jl”, and many (most?) are, this won’t work (adding that `.jl` suffix fantastically improves online search specificity so I don’t think it would be good to recommend that people not do it).

As a test I cloned Symbolics.jl and ForwardDiff.jl and got exactly the same error.

---

<div class="post-metadata">

**Author:** ![aplavin](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/aplavin/32/222056_2.png) [@aplavin](https://discourse.julialang.org/u/aplavin)\
**Post date:** [August 4, 2024, 8:31pm UTC](https://discourse.julialang.org/t/tests-with-coverage-in-vscode/117787/7 "2024-08-04T20:31:02Z")

</div>

> [@brianguenter](#):
>
> If your github project is named “\*.jl”, and many (most?) are, this won’t work (adding that `.jl` suffix fantastically improves online search specificity so I don’t think it would be good to recommend that people not do it).

Not sure how github is relevant here at all. This VSCode feature is not concerned about github, only about your local folders. And whenever you do `]dev MyPackage` in Julia, it creates a folder `~/.julia/dev/MyPackage` (not `MyPackage.jl`), just as VSCode expects.

---

<div class="post-metadata">

**Author:** ![brianguenter](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/brianguenter/32/29519_2.png) [@brianguenter](https://discourse.julialang.org/u/brianguenter)\
**Post date:** [August 4, 2024, 9:51pm UTC](https://discourse.julialang.org/t/tests-with-coverage-in-vscode/117787/8 "2024-08-04T21:51:00Z")

</div>

Different working styles I guess. I don’t use ] dev. I use the git integration built into VSCode and just clone the project from github - when you do this the directory the code clones into is named \*.jl. Or rather, it’s named exactly the name you gave it on github.

I’ve never understood the advantage of dev so haven’t used it. Everything seems to work fine without it, except for this test task which I didn’t know about till a few days ago.

Are there other features that “just work” if you use dev mode but don’t work if you just clone from github?

---

<div class="post-metadata">

**Author:** ![aplavin](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/aplavin/32/222056_2.png) [@aplavin](https://discourse.julialang.org/u/aplavin)\
**Post date:** [August 5, 2024, 6:19am UTC](https://discourse.julialang.org/t/tests-with-coverage-in-vscode/117787/9 "2024-08-05T06:19:01Z")

</div>

I’m not familiar with vscode/julia extension internals here, maybe @davidanthoff can chime in (: