Hello, I have a very basic question how to organize Julia code.
I have written several modules:
Perms.jl module Perms: ports from GAP permutation stuff. Defines functions and extends Base functions (like *, etc)
LaurentPols.jl module LaurentPols: ports from GAP Laurent polynomials. Defines stuff as above.
PermGroups.jl module PermGroups: ports from GAP permutation group stuff. uses Perms and defines stuff as above.
Cycs.jl module Cycs: ports from GAP cyclotomic numbers. Defines functions and extends Base functions (like *, etc)
Now I would like to make a package GAP including all the above stuff.
How do I write GAP.jl so using GAP will automatically imply using all the above packages
(exporting all the needed names and definitions so they can be used unqualified?)
Yes. This seems what I need. Since this facility is absent from the Julia language, does it mean the Julia creators think that packages should be organized differently? I wonder…
Nope, that’s not a reasonable inference. Reexport.jl works perfectly well as a package, so there is no reason for it to be in Julia base. One of the best features of Julia is the fact that fast generic functions and metaprogramming mean that packages can implement almost anything that Julia itself can do.
Well, there is at least one reason: documentation. The set of packages which are
clearly useful
do not have a controversial design
should be documented with the language documentation. Another example I have met is Memoize.
Otherwise, learning the language properly becomes a haphazard proposition. I have already many times since I started learning Julia 6 months ago learned via discourse something which should be in the language documentation.
If you learn something on here that you think should already be in the docs, please submit a PR introducing that thing to the docs - julia is open source and everyone is encouraged to contribute. In fact, documentation PRs are the best way to get started contributing to Julia, since you are in the unique position of having learnt something you think should be more easily accessible through the docs. The people who told you about that thing may not even know it’s not documented (although I’d like to know of an example of some core concept from the language that isn’t already in the docs).
The problem with documenting which packages are deemed “good” is that that’s a very subjective thing and has to be checked all the time, in addition to updating the docs for changes in the language. Personally, I much prefer people asking here and getting a current answer than having them ask here “I tried to do x with y as recommended by the documentation, but y is not working. Why are the docs recommending this?” as the latter is much more likely to leave a bad impression (“I don’t know how to do x” in the first case as opposed to “x should be possible with y, but it doesn’t work and now everything’s broken”).
However, I do agree that a better overview of registered packages and their use cases should be available somewhere.
Well, what I am missing most now would be a tutorial “Julia packages for dummies” which would take you by
the hand assuming only minimal or no knowledge of git and github, and would show you all the steps of making a package with a few modules, using a few other packages. Missing that I am forced to consider discourse as the only available tutorial now, in between sessions trying to understand what the manual says about packages.
I would not expect that the tutorial covers git and github. However I think one does not need to read a book
on git to write a Julia package: the barrier to write a Julia package would then be very high. I hope the necessary knowledge about git can fit in one or two pages of the tutorial.
It was never implied that reading the whole book is necessary, the book I linked has 3 intro chapters (setup, basics, branching), which would be perfectly adequate. Anything less than this may not be sufficient to make a package or contribute to code.
Also — and this may sound harsh — people who don’t want to make a modicum of investment into learning version control perhaps should not be making a Julia package, as it would quickly face a lot of maintenance issues.
The tooling that is routinely used for Julia packages may look daunting to newcomers (git, a site like Github for issue and PR management, CI, autogenerated documentation), but it is something that is relatively easy to pick up by example. Alternatively, one can wait for PkgDev to catch up, which should make all of this easier.
One could argue that CI and automated documentation are both not really a necessity for package development per se - they’re convenience tools used for automating certain dull or repetitive tasks that are no requirement for the creation of a package. As version control is a central point of managing multiple versions of packages and distributing them, I think it’s very reasonable to require a basic understanding of how it’s done. In theory, one should be able to just use any git server, register a package pointing to that server and not even deal with issues or CI via Github.
Looking at wikipedia, CI seems to be just a fancy word for “testing software before releasing it”.
This is the occasion for me to ask: the markup allows to put examples in individual functions documentation.
Already a good test is to run these examples. Is there a functionality/package to run automatically these examples as tests?
Yes, that’s precisely why I said it’s not necessary for the most basic of package development to know about this. The barrier to entry really is just git - and even of that only the most basic concepts: committing, branching, pushing to a remote - basic version control, really.
Continous Integration means having some server/service/robot automatically run specified tests on user written code - just so the developer doesn’t have to do that themselves (for example if the developer has no real access to the machine where to code will run or when building and testing the code takes a very long time).
No, and with good reason: The documentation directly above functions is not meant to be a 1:1 runnable example, but rather a quick reference for syntax, accepted arguments and the like. Having tests run automatically is exactly what CI is for - the only thing you have to do in addition to writing tests in <projectDir>/test/runtests.jl is telling e.g. Travis how to run tests for your package.
Again, this is not a requirement! One can just as easily run tests for a downloaded package manually using (v1.0) pkg> test Example (which is what I’m using right now for yet-to-be-published packages).
It seems what you’re asking for is an introduction on how to publish and maintain packages, rather than just creating them, as the latter is covered pretty well in the documentation, while the former is not.
No, and with good reason: The documentation directly above functions is not meant to be a 1:1 runnable example, but rather a quick reference for syntax, accepted arguments and the like.
If it is not possible to run automatically the examples in markup for each function, but you need to repeat them in a file runtests.jl then these examples are less useful than they could be. The need for repetition is always a sign of bad design in computer code.
In my opinion, Documentation and actually testing functionality are not very related. Documentation is plain text written for a user, to understand how to use a given function, package, etc. with a lot of (sometimes verbose) explanations. Testing and Unit Tests are just regular code which runs other code and compares outputs to some predetermined target output. They’re not meant to be read and studied. They have fundamentally different goals and are written for an entirely different reason.
Again, Examples are not Testing - they are documentation. Yes, they can be 1:1 runnable - but this does not mean that just running them will cover all possible inputs or use cases.