Is there concept of dev dependencies in Pkg?

You can create a test Project.toml, you just need to remove Test from [targets]. See also Using test/Project.toml - #4 by peterlane