Errors when testing unregistered custom package (because of non-ASCII characters, it seems)

Hello,

I created my own package (let’s call it MyPackage) with Pkg (generate, add, etc.), locally (not registered), and I have some problems to test it.

I used the latest method described here: I have a test folder inside my package root, which contains runtests.jl, Manifest.toml and Project.toml. It imports Test package (installed by ] activate test then add Test and other packages useful for tests).

Inside runtests.jl, it starts with

using Test
using MyPackage

When I am in MyPackage root folder and I run ] test, I have this error:

(MyPackage) pkg> test
    Testing MyPackage
Status `/tmp/jl_PPbTYN/Manifest.toml`
 […]
ERROR: LoadError: ArgumentError: Package MyPackage not found in current path:
- Run `import Pkg; Pkg.add("MyPackage")` to install the MyPackage package.

Stacktrace:
 [1] require(::Module, ::Symbol) at ./loading.jl:892
 [2] include(::String) at ./client.jl:439
 [3] top-level scope at none:6
in expression starting at […]/MyPackage/test/runtests.jl:11
ERROR: Package MyPackage errored during testing

Of course, my package is not registered so I can’t simply do Pkg.add("MyPackage"):

ERROR: LoadError: expected package `MyPackage […]` to be registered

And Pkg.add(".") does not work inside runtests.jl file:

ERROR: LoadError: `.` is not a valid package name

If I do ] activate test then add ., it adds MyPackage and all its dependencies, so both the Manifest.toml are pretty similar, just MyPackage/test’s has MyPackage (with repo-rev = "master" and repo-url = ".." lines in addition) and Test packages in addition, and in Project.toml, "MyPackage" = "…" line is added (with quotes, compared to other packages). In this setting, if I test again by ] test from MyPackage root folder, I have this error:

(MyPackage) pkg> test
    Testing MyPackage
ERROR: can not merge projects

It is not a version mismatch problem because I updated both MyPackage and MyPackage/test environments’ dependencies (everything is identical, except the addition of Test package in test).

My only working attempt is just to add this line inside runtests.jl (without adding MyPackage explicitely by add ., as tried in the previous step):

using Test
Pkg.activate("..")  # This line!
cd("..")  # For configuration files (is there a prettier way?), not so important for this example.
using MyPackage

So it means that by running test, it moves to test environment, activates it, imports Test, then activates the parent environment it is testing… It seems pretty odd to do that manually, especially when the Pkg documentation says for this part:

Note that Pkg will add the tested package itself implictly.

So I think I missed something…

So, my general question is: with a unregistered custom package, what is the latest and best way to test it, rather than just using Pkg.activate("..")?

Second extra question (related to a comment in the last script I have shown): is there a prettier way to manage the change of working folder for tests in regard to configuration and data files, rather than using cd("..")?

Sincerely.

at the directory where your Project.toml is located, type from bash julia --project then ]test . don’t activate test. activate means to activate your current package by reading its project toml. always start your julia at the location of Project.toml because that is where all path will be relative to when you are writing a package. no need to activate your package inside runtests.jl

I already tried that:

me@me-pc:[…]/MyPackage$ julia --project               _
   _       _ _(_)_     |  Documentation: https://docs.julialang.org
  (_)     | (_) (_)    |
   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 1.4.2 (2020-05-23)
 _/ |\__'_|_|_|\__'_|  |  Official https://julialang.org/ release
|__/                   |

(MyPackage) pkg> test
    Testing MyPackage
Status `/tmp/jl_VGlbE2/Manifest.toml`
  […]
  [c21aa512] MyPackage v0.1.0 [`[…]/MyPackage`]
  […]
ERROR: LoadError: ArgumentError: Package MyPackage not found in current path:
- Run `import Pkg; Pkg.add("MyPackage")` to install the MyPackage package.

Stacktrace:
 [1] require(::Module, ::Symbol) at ./loading.jl:892
 [2] include(::String) at ./client.jl:439
 [3] top-level scope at none:6
in expression starting at […]/MyPackage/test/runtests.jl:7
ERROR: Package MyPackage errored during testing

I activated test only to add Test and some other useful packages for tests.

follow this, at bash, cd to your Project.toml location. type julia --project. ]test

don’t change directory too in the runtests.jl. julia expects the location of test, src, docs file relative to the toplevel. you don’t have to control these locations. just start julia at the topmost level of your package directory.

show me the contents of your MyPackage.jl. I advise don’t start julia inside src directory if you are creating a package. always start it at the topmost level of the package (the one where Project.toml is located).

MyPackage.jl is is src folder, and I never starts anything from src folder.

src/MyPackage.jl (MWE):

module MyPackage

include("mysubmodule1.jl")
include("mysubmodule2.jl")
include("mysubmodule3.jl")
include("mysubmodule4.jl")

using .MySubModule1
using .MySubModule2

export start

function start()
   
end

end

that seems fine. does the test work now?

Not at all, same error as before!

For information, here are the full Project.toml files:

Project.toml:

name = "MyPackage"
uuid = "…"
authors = ["me"]
version = "0.1.0"

[deps]
Cascadia = "54eefc05-d75b-58de-a785-1a3403f0919f"
DataFrames = "a93c6f00-e57d-5684-b7b6-d8193f3e46c0"
DataStructures = "864edb3b-99cc-5e75-8d2d-829cb0a9cfe8"
FilePaths = "8fc22ac5-c921-52a6-82fd-178b2807b824"
Gumbo = "708ec375-b3d6-5a57-a7ce-8257bf98657a"
HTTP = "cd3eb016-35fb-5094-929b-558a96fad6f3"
JSON = "682c06a0-de6a-54ab-a142-c8b1cf79cde6"
LibPQ = "194296ae-ab2e-5f79-8cd4-7183a0a5a0d1"
LightXML = "9c8b4983-aa76-5018-a973-4c85ecc9e179"
Logging = "56ddb016-857b-54e1-b83d-db4d58db5568"
LoggingExtras = "e6f89c97-d47a-5376-807f-9c37f3926c36"
YAML = "ddb6d928-2868-570f-bddf-ab3f9cf99eb6"
ZipFile = "a5390f91-8eb1-5f08-bee0-b1d1ffed6cea"

test/Project.toml:

[deps]
DataFrames = "a93c6f00-e57d-5684-b7b6-d8193f3e46c0"
FilePaths = "8fc22ac5-c921-52a6-82fd-178b2807b824"
LibPQ = "194296ae-ab2e-5f79-8cd4-7183a0a5a0d1"
Pkg = "44cfe95a-1eb2-52ea-b672-e2afdf69b78f"
Test = "8dfed614-e22c-5e08-85e1-65c5234f0b40"

did you use Pkg.generate()?
uuid is missing???

Yes, I used ] generate at the very beginning of the project.

It is not missing, I just put some to avoid big texts here!

test doesn’t need a project.toml. remove it. all deps are in one project.toml

try generate a new package and work from there. something is broken with your current package. try generate a new one. run a test with some simple test file. if it works, just add your deps.

try to check some registered packages how the directories look from src, test, docs

Without test/Project.toml file, I have this error:

(MyPackage) pkg> test
    Testing MyPackage
Status `/tmp/jl_mgeehu/Manifest.toml`
[…]
  [c21aa512] MyPackage v0.1.0 [`[…]/MyPackage`]
[…]
ERROR: LoadError: ArgumentError: Package Test not found in current path:
- Run `import Pkg; Pkg.add("Test")` to install the Test package.

Of course, if I install Test again in MyPackage, same error as before!

Anyway, they say in the documentation that for Julia 1.2 and above, there are 2 Manifest and Project .toml files:

The exact interaction between Project.toml , test/Project.toml and their corresponding Manifest.toml s are not fully worked out, and may be subject to change in future versions. The old method of adding test-specific dependencies, described in the next section, will therefore be supported throughout all Julia 1.X releases.

I will try a new test project to see if it happens again…

I already checked several packages, but because they are registered, it is not the same problem!

add Test at the package prompt

the error is that you did not add the Test as deps

Look my edition: it comes back to the previous error…

Project.toml should have Test as part of deps. add any package it needs but not inside test/Project.toml. they should be added at the main Project.toml

And in this case, I have the same error:

ERROR: LoadError: ArgumentError: Package MyPackage not found in current path:
- Run `import Pkg; Pkg.add("MyPackage")` to install the MyPackage package.

So, if Test package is inside the unique MyPackage/Project.toml file, I have this error, same as when there is also a MyPackage/test/Project.toml file with Test package there, and also same as when Test is in both files if there are 2 Project.toml files.

Sorry for other readers, we were talking at the same time!

OK, it seems I found the problem, which was impossible to find here because I made it easier to read for readers. It seems it is related to a non-ASCII character in the name of my package…

I am doing comparison test to verify.