# Different requirements for packages, using, building, and testing

**URL:** <https://discourse.julialang.org/t/different-requirements-for-packages-using-building-and-testing/2170>\
**Category:** General Usage\
**Tags:** package\
**Created:** [February 18, 2017, 10:58am UTC](https://discourse.julialang.org/t/different-requirements-for-packages-using-building-and-testing/2170 "2017-02-18T10:58:38Z")\
**Posts on this page:** 16\
**Page:** 1

<div class="post-metadata">

**Author:** ![ScottPJones](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/scottpjones/32/146_2.png) [@ScottPJones](https://discourse.julialang.org/u/ScottPJones)\
**Post date:** [February 18, 2017, 10:58am UTC](https://discourse.julialang.org/t/different-requirements-for-packages-using-building-and-testing/2170/1 "2017-02-18T10:58:38Z")

</div>

Is there a way of specifying distinct requirements for a package, for using it, for building, and for testing?

For example, I have a package that needs LightXML, but only for running deps/build.jl, which downloads one or more data files, stores them in a data subdirectory, and then produces a data file which it also stores there, which is used by the package. The package itself does not need LightXML, and I don’t want the extra overhead of that dependency just to use the package. The same is true for testing, where we want BaseTestNext (for v0.4.7) and BenchmarkTools, but only when running the unit tests, not for deployment.

If there is no way of handling those issues, will that be something addressed by the Pkg3 redesign?  
Also, is it acceptable for the build.jl or runtests.jl to check to see if the packages(s) are already present, and if not,  
call Pkg.add (and possibly Pkg.build), before `using` the package(s), in order to keep those out of REQUIRES?

---

<div class="post-metadata">

**Author:** ![lobingera](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lobingera/32/211_2.png) [@lobingera](https://discourse.julialang.org/u/lobingera)\
**Post date:** [February 18, 2017, 11:24am UTC](https://discourse.julialang.org/t/different-requirements-for-packages-using-building-and-testing/2170/2 "2017-02-18T11:24:11Z")

</div>

You can have REQUIRE in test additionally.  
Packages should - as far as i understood some discussions in the METADATA PRs - never run Pkg.add on their own.

And i agree with you, all this can take you to dependency hell.

---

<div class="post-metadata">

**Author:** ![ScottPJones](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/scottpjones/32/146_2.png) [@ScottPJones](https://discourse.julialang.org/u/ScottPJones)\
**Post date:** [February 18, 2017, 12:53pm UTC](https://discourse.julialang.org/t/different-requirements-for-packages-using-building-and-testing/2170/3 "2017-02-18T12:53:41Z")

</div>

Thanks! That will help some.

You can’t have a separate REQUIRE in deps? That’s the one that causes the most problems.

---

<div class="post-metadata">

**Author:** ![lobingera](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lobingera/32/211_2.png) [@lobingera](https://discourse.julialang.org/u/lobingera)\
**Post date:** [February 19, 2017, 11:23am UTC](https://discourse.julialang.org/t/different-requirements-for-packages-using-building-and-testing/2170/4 "2017-02-19T11:23:56Z")

</div>

Could you explain, what problem you actually try to solve?  
The deps/ dir is part of the BinDeps setup, so you should find help for installation specifics there first. Then i don’t see LightXML as heavy dependency for building a package, but if you disagree why not commiting the data file directly to the package?  
And i have some problems to understand the logic: To run/use a package it needs to be build locally, so in this case LightXML is a valid, required dependency. To have additional requirements for testing, makes some sense, as not everyone needs to run testing on installed packages, but everyone runs a build.

---

<div class="post-metadata">

**Author:** ![ScottPJones](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/scottpjones/32/146_2.png) [@ScottPJones](https://discourse.julialang.org/u/ScottPJones)\
**Post date:** [February 19, 2017, 12:25pm UTC](https://discourse.julialang.org/t/different-requirements-for-packages-using-building-and-testing/2170/5 "2017-02-19T12:25:42Z")

</div>

I’m not using BinDeps at all (that’s for _binary_ dependecies AFAIK, it just hooks into the general deps/build.jl setup), `deps/build.jl` is called by `Pkg.build`.

LightXML (for one of the packages) and JSON (for some of the others) are only needed for building the data tables, after that point, they are not needed at all to use the package.  
The data tables only need to be rebuilt when something changes, like a new release of Unicode, say when 10.0 comes out, or if somebody wants to manually add an entity to the LaTeX entity tables. (I need to add that capability to manually add entities to the other Emoji and HTML entity tables as well).

I suppose I can check in the current state of the data tables, and rename deps/build.jl to something else, so that it won’t get called by Pkg.build() automatically, and document how it can be called to rebuild the data table.

---

<div class="post-metadata">

**Author:** ![lobingera](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lobingera/32/211_2.png) [@lobingera](https://discourse.julialang.org/u/lobingera)\
**Post date:** [February 19, 2017, 2:16pm UTC](https://discourse.julialang.org/t/different-requirements-for-packages-using-building-and-testing/2170/6 "2017-02-19T14:16:06Z")

</div>

> [@ScottPJones](#):
>
> LightXML (for one of the packages) and JSON (for some of the others) are only needed for building the data tables, after that point, they are not needed at all to use the package.

Do you assume that it’s possible to use/run a package without building it?

---

<div class="post-metadata">

**Author:** ![ScottPJones](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/scottpjones/32/146_2.png) [@ScottPJones](https://discourse.julialang.org/u/ScottPJones)\
**Post date:** [February 19, 2017, 2:29pm UTC](https://discourse.julialang.org/t/different-requirements-for-packages-using-building-and-testing/2170/7 "2017-02-19T14:29:55Z")

</div>

The build script downloads the input data files (for Unicode, HTML, Emojis, or LaTeX entities respectively, for each package) if they are not already present, and processes them to produce the data file that is needed when the package is used/run. None of that is needed for deploying the package, just the final output data file.

In the emoji\_symbols.jl and latex\_symbols.jl files, the scripts used to create them (which are out of data and no longer work) were included as comments in the output files (instead of producing compact data files, they produce julia code that needs to be compiled and run, which adds a lot of extra time / unreclaimable space (since julia code, AFAIK, is not GCed).

What I’m trying to do is make the cost of having loadable tables (which also allow one to update them between releases) much smaller, by separating out the process of building the tables and creating the Julia structures, without requiring unnecessary run-time dependencies.

---

<div class="post-metadata">

**Author:** ![djsegal](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/djsegal/32/13752_2.png) [@djsegal](https://discourse.julialang.org/u/djsegal)\
**Post date:** [February 19, 2017, 4:23pm UTC](https://discourse.julialang.org/t/different-requirements-for-packages-using-building-and-testing/2170/8 "2017-02-19T16:23:37Z")

</div>

Isn’t this what got discussed in: [Development, Test, and Production Environments](https://discourse.julialang.org/t/development-test-and-production-environments/1974)?

with it you can do something like this for REQUIRE:

```julia
gem 'httparty'

group :development, :test do
  gem 'better_errors'
end

group :production do
  gem 'postgres'
end

```

// edit: i just used a gemfile instead of a require file because i had one on hand

---

<div class="post-metadata">

**Author:** ![kdheepak](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/kdheepak/32/10881_2.png) [@kdheepak](https://discourse.julialang.org/u/kdheepak)\
**Post date:** [July 31, 2018, 6:09pm UTC](https://discourse.julialang.org/t/different-requirements-for-packages-using-building-and-testing/2170/9 "2018-07-31T18:09:33Z")

</div>

I have a similar issue that I would like to solve. The package that I’m developing does not use the dependency InfoZip. However the tests require downloading some data and I require using InfoZip to run the tests.

I have a similar need in another where `Pkg.build` adds additional data optionally, that is not required for the main package to run. Is there a way for specifying build requirements, test requirements and package run time requirements separately?

---

<div class="post-metadata">

**Author:** ![fredrikekre](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/fredrikekre/32/1688_2.png) [@fredrikekre](https://discourse.julialang.org/u/fredrikekre)\
**Post date:** [July 31, 2018, 6:13pm UTC](https://discourse.julialang.org/t/different-requirements-for-packages-using-building-and-testing/2170/10 "2018-07-31T18:13:16Z")

</div>

Julia 0.6:  
[https://docs.julialang.org/en/stable/manual/packages/#REQUIRE-speaks-for-itself-1](https://docs.julialang.org/en/stable/manual/packages/#REQUIRE-speaks-for-itself-1)

Julia 0.7:  
[https://docs.julialang.org/en/latest/stdlib/Pkg/#Test-specific-dependencies-1](https://docs.julialang.org/en/latest/stdlib/Pkg/#Test-specific-dependencies-1)

---

<div class="post-metadata">

**Author:** ![kdheepak](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/kdheepak/32/10881_2.png) [@kdheepak](https://discourse.julialang.org/u/kdheepak)\
**Post date:** [July 31, 2018, 6:50pm UTC](https://discourse.julialang.org/t/different-requirements-for-packages-using-building-and-testing/2170/11 "2018-07-31T18:50:06Z")

</div>

Thanks! That was helpful! I will try that out.

---

<div class="post-metadata">

**Author:** ![kdheepak](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/kdheepak/32/10881_2.png) [@kdheepak](https://discourse.julialang.org/u/kdheepak)\
**Post date:** [July 31, 2018, 7:45pm UTC](https://discourse.julialang.org/t/different-requirements-for-packages-using-building-and-testing/2170/12 "2018-07-31T19:45:48Z")

</div>

@fredrikekre What about build dependencies? If I have a requirement for `Pkg.build` that’s not part of the main package what should I do. I essentially want to download and unzip a large zip file with a lot of data before running any tests.

```julia
Pkg.clone("https://github.com/Org/Package")
Pkg.build("Package") # fails because dependency wasn't installed
Pkg.test("Package") # Installs dependency

```

Suggestions?

---

<div class="post-metadata">

**Author:** ![oxinabox](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/oxinabox/32/206603_2.png) [@oxinabox](https://discourse.julialang.org/u/oxinabox)\
**Post date:** [August 1, 2018, 12:30am UTC](https://discourse.julialang.org/t/different-requirements-for-packages-using-building-and-testing/2170/13 "2018-08-01T00:30:05Z")

</div>

`Pkg.build` is run by everyone installing the package – it needs to be run before the package can be used.  
(unlike `Pkg.test`).  
thus anything you need for building, is simply something you need.

So it goes in the main dependencies.

---

<div class="post-metadata">

**Author:** ![StefanKarpinski](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stefankarpinski/32/24_2.png) [@StefanKarpinski](https://discourse.julialang.org/u/StefanKarpinski)\
**Post date:** [August 1, 2018, 4:04am UTC](https://discourse.julialang.org/t/different-requirements-for-packages-using-building-and-testing/2170/14 "2018-08-01T04:04:23Z")

</div>

That’s not entirely true: build deps need to be present at build time but don’t need to remain installed and even more importantly don’t need to be loaded into the same Julia process so they don’t need to be compatible with all of a project’s dependencies. You can add a `build` target the same way you can add a `test` target.

---

<div class="post-metadata">

**Author:** ![oxinabox](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/oxinabox/32/206603_2.png) [@oxinabox](https://discourse.julialang.org/u/oxinabox)\
**Post date:** [August 1, 2018, 4:40am UTC](https://discourse.julialang.org/t/different-requirements-for-packages-using-building-and-testing/2170/15 "2018-08-01T04:40:18Z")

</div>

That is cool!

Can I have arbitrary targets, keyed to any command run by `Pkg`?

Could I have say a `dev` target that is for dependencies needed during development.  
For example static code generation, the output of which is checked into version control.

Or is it just build and test?

---

<div class="post-metadata">

**Author:** ![StefanKarpinski](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stefankarpinski/32/24_2.png) [@StefanKarpinski](https://discourse.julialang.org/u/StefanKarpinski)\
**Post date:** [August 1, 2018, 4:57am UTC](https://discourse.julialang.org/t/different-requirements-for-packages-using-building-and-testing/2170/16 "2018-08-01T04:57:47Z")

</div>

You can have any kind of target: they’re named subsets of deps + extras (which must include deps). The test and build targets happen have special meaning to the package manager. In the future we might have other ones that it knows about but the mechanism for expressing them is the same.