Suggestion for a new package: MinimalWorkingExamples.jl

FWIW, I don’t find this that much of a hassle, unless the MWE uses a lot of packages — in practice, most of them just use 1–5, more is kind of uncommon and should live in a repo with a manifest.

For the latter, Github Gist actually works fine, because multiple files can exist in the same gist and be downloaded as a zip file. If a Manifest.toml is included, downloading, extracting, instantiating and activating that is a reasonable and convenient workflow.

Compare what we do today (ignore the errors):
mwe

with this:

mwe2

(it looks better if you click on the gifs)
I just added using MinimumWorkingExample to your MWE.

3 Likes

This example underscores my point above: you really need a manifest for anything nontrivial. Note how the code in that example is outdated now (will fail at CSV.Source).

I think it would be better to come up with a convenience solution that reproduces an environment in a few steps, rather than just automatically downloading the latest versions of some packages. And, from the user side, that packs code + manifest to an archive or a gist.

2 Likes

I do wish there was some sort of environment or REPL mode for what you are describing. It would be great if using MinimumWorkingExample would activate a temp environment and also detect and add packages that were subsequently entered by using ThisPackage.

And to get at @Tamas_Papp point, perhaps what this MinimumWorkingExample.jl should do is package the MWE so that other people can run it. So it would have dual purposes:

  1. Users creating an MWE would using MinimumWorkingExamples and call a function on a .jl script from which they would like to create an MWE. This package would create a reproducible environment and push it … somewhere (GitHub Gist? making this up as I go, sorry). It generates a link that they can share.
  2. Users wanting to run an MWE would using MinimumWorkingExamples and call a function on some provided link which would then download and execute the code. I guess it would be most beneficial if it also opened the code in your editor or something.

Thinking out loud here.

2 Likes

@tbeason, ooo, I like that. Very cool!

And @Tamas_Papp, yea, I guess my main point is that (to begin with at least) this would only be for immediately testing someone’s MWE. Failing an outdated MWE or a complicated one is totally fine. I guess the question is: Are these small, simple, and ephemeral MWE too simple for building and using such a tool, or not?

Just throwing in some additional points that may benefit the creator of an MWE (I’d agree with Tamas in that it’s usually simple enough to run MWEs)

  • Assert self-containment: you often enough see MWE that are missing packages, variable or function definitions and the like. A useful functionality would be something like a macro that you could wrap around your mwe that would throw if the example couldn’t be run on its own

  • Environment state: it may be as simple as outputting or prepending to an existing example a series of

using Pkg
Pkg.activate(...) # temp? 
Pkg.add(...) # pinned to the exact versions that are at the environment at the time of writing the mwe
... 
using...

This would also make it completely self-contained and you don’t need another package to run it.

  • Up-to-dateness: issuing a warning when you create a mwe with outdated packages in the environment. I know it should not be the task of such a package, but it would catch quite a lot of questions before they even appear here (looking at DataFrames, surprisingly many people end up with v0.19 something of that)

  • some debut info is never bad. If the package would pack an mwe into a neat self-contained script, it could likewise append some basic information as comments at the end (especially OS, build and Julia version)

  • should be easy to discover and use. Maybe just two entry points, a macro for in-script or REPL usage (@mwe begin... mwe... end) that would generate a clean mwe.jl script and a function that would execute the above checks for an existing script

  • promotional header: the first line of a generated script should be a comment stating that it was generated with MWE.jl for two reasons: If you intend to run the mwe to offer help, you immediately know that the most basic checks were already done and you can just copy paste everything. And additionally, new users and people searching for help in the forum see that there’s a package for that stuff.

3 Likes

Some awesome stuff. Allow me to summarize:

I think we can agree, there are two different tools being discussed here:

  1. minimal tool that just alleviates the need to activate a temp env and add the packages mentioned in small, minimal, and ephemeral MWE.
  2. robust tool that guarantees the up-to-dateness, completeness, and reproducibility of any MWE.

The first simpler tool would save some typing when testing people’s MWE. The second tool would dramatically increase the quality of MWE in the whole ecosystem. And as @FPGro mentioned, it might solve some of the issues people see just by checking the up-to-dateness and completeness of people’s MWE. I mean, it could revolutionize MWE in Julia forever :slight_smile:

So while I started this off with the idea of something small but useful (tool #1) I’m slowly realizing that the second robust tool would have a much bigger positive effect on the community. Obviously, the robust tool would also alleviate the same pains the first tool was supposed to address, so win win!

1 Like

I still think you need to work in a different process with a clean load path/depot, otherwise you stack on the global environment (and people tend to have huge global environments), which isn’t good.

I totally agree. Not doing that would prevent us from being able to reproduce the exact same environment (baring system differences).

This movie makes it clear for me, I hadn’t fully grasped your idea. I pretty much would love to have something like that!

1 Like

@fredrikekre pointed out on Zulip that you can activate a completely clean environment (with empty load path) with

push!(empty!(LOAD_PATH), joinpath(mktempdir(), "Project.toml"))

without having to start a separate Julia process.

3 Likes