# Best practices for examples with Pkg3

**URL:** https://discourse.julialang.org/t/best-practices-for-examples-with-pkg3/13356
**Category:** General Usage
**Created:** [August 13, 2018, 11:17am UTC](https://discourse.julialang.org/t/best-practices-for-examples-with-pkg3/13356 "2018-08-13T11:17:51Z")
**Posts on this page:** 16
**Page:** 1

<div class="post-metadata">

### Author: ![krcools](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/krcools/32/10212_2.png) [@krcools](https://discourse.julialang.org/u/krcools)
#### Post date: [August 13, 2018, 11:17am UTC](https://discourse.julialang.org/t/best-practices-for-examples-with-pkg3/13356/1 "2018-08-13T11:17:51Z")

</div>

Often, example scripts explicitly import dependencies of the package they are included with at top level scope. This was not an issue with OldPkg as the dependencies are installed and visible for code loading in any context.

With Pkg(3), the dependencies need to be explicitly added to the active environment. This requires more actions and skills from your audience and might increase the threshold and decrease willingness of people to explore your package and features.

What is recommended in this scenario? Related, what is the best/portable way to refer to the `examples` directory from the REPL?

---

<div class="post-metadata">

### Author: ![Tamas\_Papp](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tamas_papp/32/25949_2.png) [@Tamas\_Papp](https://discourse.julialang.org/u/Tamas_Papp)
#### Post date: [August 13, 2018, 12:55pm UTC](https://discourse.julialang.org/t/best-practices-for-examples-with-pkg3/13356/2 "2018-08-13T12:55:31Z")

</div>

I am not sure I understand the question, but my understanding of the difference is that previously you had to specify dependencies in `REQUIRE` which was used for installation, but not checked when loading, while for the new `Pkg` you have to declare the dependency before loading.

For package developers, AFAIK the recommended workflow is to

```julia
pkg> activate path/to/package
pkg> add ThisDependency

```

which IMO is not that much of a barrier to entry.

---

<div class="post-metadata">

### Author: ![krcools](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/krcools/32/10212_2.png) [@krcools](https://discourse.julialang.org/u/krcools)
#### Post date: [August 13, 2018, 1:15pm UTC](https://discourse.julialang.org/t/best-practices-for-examples-with-pkg3/13356/3 "2018-08-13T13:15:45Z")

</div>

Sorry for not being clearer. Say `Foo` has a dependency `Bars`. Perhaps the API of `Foo` includes a method

```julia
Foo.f(x::Bars.Bar) = (x,x)

```

I understand that I - as a package developer - have to specify `Bars` as a dependency by declaring it in either `REQUIRE` or `Project.toml`. To explain to my users how the package is used I include an example at `Foo/examples/hello.jl`:

```julia
using Foo, Bars
x = Bar(3)
xx = f(x)

```

In Julia-0.6, users could run this by e.g.:

```julia
Pkg.add("Foo")
include(joinpath(Pkg.dir("Foo"),"/examples/hello.jl"))

```

In julia-0.7 this fails because only `Foo` can be loaded in scripts included at the REPL. To solve this one would have to warn the user that additional installs are required to run the examples (in `README.md`? As comments in `hello.jl`?) Users might miss this and will be confronted with package manager errors when trying out the example.

I guess a solution could be to put each example in a separate directory with its own ‘Project.toml’ file. Still you would have to instruct the user to:

- Navigate to that directory (not trivial given the slug in the path)
- `]activate .`
- `]instantiate`
- `include("hello.jl")`

EDIT: Another solution would be to always reexport all symbols from imported dependencies. But that feels like it defeats the point of splitting up functionality over different packages.

---

<div class="post-metadata">

### Author: ![Tamas\_Papp](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tamas_papp/32/25949_2.png) [@Tamas\_Papp](https://discourse.julialang.org/u/Tamas_Papp)
#### Post date: [August 13, 2018, 1:25pm UTC](https://discourse.julialang.org/t/best-practices-for-examples-with-pkg3/13356/4 "2018-08-13T13:25:55Z")

</div>

In the global/default environment,

```julia
pkg> add git/repofor/Foo

```

for unregistered packages should be sufficient, then the user can run whatever script is required.

Also note that `Pkg.dir` is gone, search this forum for the relevant discussions.

---

<div class="post-metadata">

### Author: ![krcools](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/krcools/32/10212_2.png) [@krcools](https://discourse.julialang.org/u/krcools)
#### Post date: [August 13, 2018, 1:35pm UTC](https://discourse.julialang.org/t/best-practices-for-examples-with-pkg3/13356/5 "2018-08-13T13:35:54Z")

</div>

Sorry but still not what I am steering at.

I understand that `Pkg.dir` is gone, this is part of what makes it harder for users playing around with packages and their included example (I understand there is an unexported and undocumented `Base.find_package` that alleviates this to some extent?).

Your oneliner indeed should install the package `Foo` and its dependency `Bars`. The problem is that `Bar` will only be visible from within `Foo`, not from the top-level examples that should provide a gentle getting started guide to the user.

In other words what is the minimum number of manipulations that makes this work (lets assume both `Foo` and `Bars` are registered:

```julia
pkg>add Foo
<stuff>
julia>include(<morestuff>, "hello.jl")

```

Ideally I do not want to burden my visitor with inspecting what `hello.jl` depends on and installing those dependencies (which is the case in julia-0.6).

---

<div class="post-metadata">

### Author: ![Tamas\_Papp](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tamas_papp/32/25949_2.png) [@Tamas\_Papp](https://discourse.julialang.org/u/Tamas_Papp)
#### Post date: [August 13, 2018, 1:40pm UTC](https://discourse.julialang.org/t/best-practices-for-examples-with-pkg3/13356/6 "2018-08-13T13:40:38Z")

</div>

`<stuff>` = `pkg> add Bar`

As for `<morestuff>`,

1. define `Foo.examplepath(...)` using `@ __DIR__ ` (recommended solution),
2. use `Base.pathof(Foo)`.

---

<div class="post-metadata">

### Author: ![krcools](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/krcools/32/10212_2.png) [@krcools](https://discourse.julialang.org/u/krcools)
#### Post date: [August 13, 2018, 1:55pm UTC](https://discourse.julialang.org/t/best-practices-for-examples-with-pkg3/13356/7 "2018-08-13T13:55:48Z")

</div>

This is a solution but you will agree it adds a significant overhead compared to the steps required in Julia-0.6. I would argue your solution expands to:

- Read `hello.jl`, or let’s call it `demo_all_pkg_features.jl` to stress there are cases in which the user is not necessarily interested in the details of the code contained in the example.
- Correctly list and add all dependencies. Users might find this difficult or might be unwilling to pollute their primal environment.
- Then after all installations complete:

```julia
using Foo # yet another step not required in 0.6
d = include(Foo.examplepath("hello.jl"))

```

Given how lightweight projects and environments have become I was really hoping for a much more idiomatic solution to this very common problem.

---

<div class="post-metadata">

### Author: ![Tamas\_Papp](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tamas_papp/32/25949_2.png) [@Tamas\_Papp](https://discourse.julialang.org/u/Tamas_Papp)
#### Post date: [August 13, 2018, 2:09pm UTC](https://discourse.julialang.org/t/best-practices-for-examples-with-pkg3/13356/8 "2018-08-13T14:09:39Z")

</div>

> [@krcools](#):
>
> you will agree it adds a significant overhead compared to the steps required in Julia-0.6

Nope, I don’t agree.

> [@krcools](#):
>
> Read `hello.jl` , or let’s call it `demo_all_pkg_features.jl` to stress there are cases in which the user is not necessarily interested in the details of the code contained in the example.

Sorry, I don’t understand what you are saying here. If the user is not interested in the example code, then what purpose does it serve?

> [@krcools](#):
>
> Correctly list and add all dependencies. Users might find this difficult or might be unwilling to pollute their primal environment.

Again, installing a package is just a single `add` command. `Pkg` will install the dependencies. In case the example requires an additional package, that should be installed, too. I think this is trivial.

Also, I don’t really see the issue about “pollution” of the environment, but if the user is concerned about that, they should `activate` another one.

Finally, I don’t see the point of `include`ing an example. I would just open it in the editor, and send it to the REPL line by line to understand what is going on.

---

<div class="post-metadata">

### Author: ![krcools](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/krcools/32/10212_2.png) [@krcools](https://discourse.julialang.org/u/krcools)
#### Post date: [August 13, 2018, 2:37pm UTC](https://discourse.julialang.org/t/best-practices-for-examples-with-pkg3/13356/9 "2018-08-13T14:37:17Z")

</div>

> Sorry, I don’t understand what you are saying here. If the user is not interested in the example code, then what purpose does it serve?

Someone might be looking for a good plotting package. Often the example showcase what type of plots are in store. At this point I just want to know whether this package can give me a histogram with transparent bars. I don’t want to worry about whether or not I need e.g. `GeomTypes` whilst shopping for a plotting lib. Just running examples can also give a good idea w.r.t to the package performance.

In `GLVisualize`, the authors support prospective users by leveraging the testing infrastructure of Julia. In this case, a oneliner:

```julia
pkg>test GLVisualize

```

will resolve additional dependencies (and expose them top-level if needed), create a separate environment, instantiate it, and include the tests (or interactive demos as is the case for `GLVisualize`).

In my opinion it would be convenient for this functionality to be extended to something like:

```julia
pkg>example Foo hello

```

I agree just running the examples it too limiting. Maybe in addition to what happens when testing, the examples could be copied over in a directory with a predictable name (or custom name), similar to what happens when you `develop` a package.

Given the flexibility of the new package manager and the how common it is for packages to ship with an examples subdirectory I was just wondering whether someone had automated something like this.

---

<div class="post-metadata">

### Author: ![Tamas\_Papp](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tamas_papp/32/25949_2.png) [@Tamas\_Papp](https://discourse.julialang.org/u/Tamas_Papp)
#### Post date: [August 13, 2018, 3:17pm UTC](https://discourse.julialang.org/t/best-practices-for-examples-with-pkg3/13356/10 "2018-08-13T15:17:46Z")

</div>

> [@krcools](#):
>
> Often the example showcase what type of plots are in store. At this point I just want to know whether this package can give me a histogram with transparent bars.

Presumably the documentation or the main page of the project on Github etc would be a good starting places for this information. Eg [PGFPlotsX does this](https://kristofferc.github.io/PGFPlotsX.jl/latest/examples/gallery.html).

However, if you want code in a directly executable form, probably packaging the demo itself as an additional project would be a viable solution.

---

<div class="post-metadata">

### Author: ![StefanKarpinski](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stefankarpinski/32/24_2.png) [@StefanKarpinski](https://discourse.julialang.org/u/StefanKarpinski)
#### Post date: [August 13, 2018, 4:19pm UTC](https://discourse.julialang.org/t/best-practices-for-examples-with-pkg3/13356/11 "2018-08-13T16:19:45Z")

</div>

> [@krcools](#):
>
> I understand there is an unexported and undocumented `Base.find_package` that alleviates this to some extent?

You do this:

```julia
julia> using StaticArrays
[Info: Precompiling StaticArrays [90137ffa-7385-5640-81b9-e52037218182]

julia> pathof(StaticArrays)
"/Users/stefan/.julia/packages/StaticArrays/Ze5H3/src/StaticArrays.jl"

```

The difference is that you have to pass the package module rather than its name.

---

<div class="post-metadata">

### Author: ![StefanKarpinski](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stefankarpinski/32/24_2.png) [@StefanKarpinski](https://discourse.julialang.org/u/StefanKarpinski)
#### Post date: [August 13, 2018, 4:25pm UTC](https://discourse.julialang.org/t/best-practices-for-examples-with-pkg3/13356/12 "2018-08-13T16:25:30Z")

</div>

> [@Tamas\_Papp](#):
>
> Also, I don’t really see the issue about “pollution” of the environment, but if the user is concerned about that, they should `activate` another one.

Moreover, the targets mechanism makes it easy for the package to declare project environments that extend its own, just do:

```toml
[extras]
Bars = "<uuid>"

[targets]
examples = ["Bars"]

```

We still need an API to activate target environments (`pkg> activate Bars.examples`)?, but with that you could just do that and you’ll be able to run the examples from that environment easily.

---

<div class="post-metadata">

### Author: ![StefanKarpinski](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stefankarpinski/32/24_2.png) [@StefanKarpinski](https://discourse.julialang.org/u/StefanKarpinski)
#### Post date: [August 13, 2018, 4:52pm UTC](https://discourse.julialang.org/t/best-practices-for-examples-with-pkg3/13356/13 "2018-08-13T16:52:25Z")

</div>

At a very high level, this issue is about whether `Bars` is an internal implementation detail of `Foo` or if it is part of `Foo`’s external API. In general, one does not want a package’s dependencies exposed to its users since doing that prevents it from changing its implementation in the future. The new package manager and code loading mechanism takes a very hard stance on this: dependencies are an internal implementation detail and are private. To the point that one of your dependencies can have a dependency with the same name as one of your project’s dependencies even if it is actually a different package.

Running package examples is definitely something we need to work out how to do conveniently, but it does not take precedence in importance over keeping a package’s internal implementation details isolated and private. I’m not saying that to chastise but to frame this conversation appropriately: rather than complaining about / trying to redesign the code loading so that this one relatively marginal thing can be done as easily as possible, try to think about how to make running example code easy, taking the way code loading now works as a fixed point—since it is not changing now that 1.0 is released.

---

<div class="post-metadata">

### Author: ![baggepinnen](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/baggepinnen/32/693_2.png) [@baggepinnen](https://discourse.julialang.org/u/baggepinnen)
#### Post date: [August 14, 2018, 6:20am UTC](https://discourse.julialang.org/t/best-practices-for-examples-with-pkg3/13356/14 "2018-08-14T06:20:30Z")

</div>

Is the strategy to re-export relevant parts of `Bars` from `Foo` a reasonable alternative, considering that users of `Foo` are expected to know about and use `Bars`? Or this is no longer possible the way it was in 0.6?

If it is a reasonable strategy, is [Reexport.jl](https://github.com/simonster/Reexport.jl) a good option? I see it has been updated recently to support Julia 1.0, but have heard some loose arguments against use of it before.

---

<div class="post-metadata">

### Author: ![krcools](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/krcools/32/10212_2.png) [@krcools](https://discourse.julialang.org/u/krcools)
#### Post date: [August 14, 2018, 7:29am UTC](https://discourse.julialang.org/t/best-practices-for-examples-with-pkg3/13356/15 "2018-08-14T07:29:25Z")

</div>

I very much welcome the strictness of the package manager. What I was looking for was something built on top of the current framework, like the solution you mention is in the works.

I don’t think the situation I described necessarily comes from `Foo` exposing its implementation. The situation I found myself in is with `Foo` a library that is designed to do processing on objects implementing the Bar concept (concept as in the c++ meaning of the word). Package `Bars` defines the concept by (i) introducing a set of functions meant for other packages to extend, and (ii) providing a reference implementation `Bar` accompanied by unit tests that guard the semantics of what it means to be a Bar.

A bad example of this is the Array concept in `Base`. `Base` introduces the _original_ `getindex` etc, but also includes the reference implementation `Array{T,N}`. As a result, many other packages both import `Base` and allow objects from `Base` to be passed to the API. This example is bad because `Base` is the one package the manager lets you have for free.

As another example I could think of a package `Fields` defining the mathematical concept of field by introducing say `mult_inverse` etc. and providing the reference implementation `Galois{P}`. Then another package `LinearSpaces` comes along that will have to import functions from `Fields` to implement say `elementwise_addition`. It is very like `LinearSpaces` will also include `Fields` in its suite of examples. I wouldn’t say in this case the packages needlessly expose implementation details.

---

<div class="post-metadata">

### Author: ![jdlara-berkeley](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jdlara-berkeley/32/4096_2.png) [@jdlara-berkeley](https://discourse.julialang.org/u/jdlara-berkeley)
#### Post date: [August 24, 2018, 11:20pm UTC](https://discourse.julialang.org/t/best-practices-for-examples-with-pkg3/13356/16 "2018-08-24T23:20:03Z")

</div>

Since `test` can be defined as a target, how about `build`? In some cases, one could need some packages, specially Pkg itself just for building and wouldn’t want to include it as a dependency in the project.
