[ANN] PatModules.jl: a better module system for Julia

Package Announcements
58

I suggest looking at the following as well:
→ Dependencies of src files inside a package

I also creating a doc PR to address this issue (I don’t suggest reading the PR thread because it’s a bit hard to read).

Instead, I’ll insert the PR text directly here:

Inline text from PR

Julia ⇔ C/C++: Namespaces

  • C/C++ namespaces correspond roughly to Julia modules.
  • There are no private functions/variables/modules/… in Julia. Everthing is accessible
    through fully qualified paths (or relative paths, if desired).
  • using MyNamespace::myfun (C++) corresponds roughly to import MyModule: myfun (Julia).
  • using namespace MyNamespace (C++) corresponds roughly to using MyModule (Julia)
    • In Julia, only exported symbols are made available to the calling module.
    • In C++, only elements found in the included (public) header files are made available.
  • Caveat: import/using keywords (Julia) also load modules (see below).
  • Caveat: import/using (Julia) works only at the global scope level (modules)
    • In C++, using namespace X works within arbitrary scopes (ex: function scope).

Julia ⇔ C/C++: Module loading

  • When you think of a C/C++ “library”, you are likely looking for a Julia “package”.
    • Caveat: C/C++ libraries often house multiple “software modules” whereas Julia
      “packages” typically house one.
    • Reminder: Julia modules are global scopes (not necessarily “software modules”).
  • Instead of build/make scripts, Julia uses “Project Environments” (sometimes called
    either “Project” or “Environment”).
    • Build scripts are only needed for more complex applications
      (like those needing to compile, or download C/C++ executables :slight_smile: ).
    • C/C++ code typically target more conventional applications, whereas Julia
      “Project Environments” provide a set of packages to experiment with particular problem
      spaces. Julia users typically use problem-specific “scripts” for this type of experimentation.
    • To develop a “conventional” application/project in Julia, you can initialize its root directory
      as a “Project Environment”, and house application-specific code/packages there.
      This provides good control over project dependencies, and future reproducibility.
    • Available packages are added to a “Project Environment” with the pkg> add tool
      (This does not load said package, however).
    • The list of available packages (direct dependencies) for a “Project Environment” are
      saved in its Project.toml file.
    • The full dependency information for a “Project Environment” is auto-generated & saved
      in its Manifest.toml file.
  • Packages (“software modules”) available to the “Project Environment” are loaded with
    import or using.
    • In C/C++, you #include <moduleheader> to get object/function delarations, and link in
      libraries when you build the executable.
    • In Julia, whatever is loaded is available to all other loaded modules through its
      fully qualified path (no header file required).
    • Use import SomePkg: SubModule.SubSubmodule (Julia) to access package submodules.
  • Directory-based package repositories (Julia) can be made available by adding repository
    paths to the Base.LOAD_PATH array.
    • Packages from directory-based repositories do not require the pkg> add tool prior to
      being loaded with import or using. They are simply available to the project.
    • Directory-based package repositories are the quickest solution to developping local
      libraries of “software modules”.

Julia ⇔ C/C++: Assembling modules

  • In C/C++, .c/.cpp files are compiled & added to a library with build/make scripts.
    • In Julia, import [PkgName]/using [PkgName] statements load [PkgName].jl located
      in a package’s [PkgName]/src/ subdirectory.
    • In turn, [PkgName].jl typically loads associated source files with calls to
      include "[someotherfile].jl".
  • include "./path/to/somefile.jl" (Julia) is very similar to
    #include "./path/to/somefile.jl" (C/C++).
    • However include "..." (Julia) is not used to include header files (not required).
    • Do not use include "..." (Julia) to load code from other “software modules”
      (use import/using instead).
    • include "path/to/some/module.jl" (Julia) would instantiate multiple versions of the
      same code in different modules (creating distinct types (etc.) with the same names).
    • include "somefile.jl" is typically used to assemble multiple files within the same
      Julia package       (“software module”). It is therefore relatively straightforward to ensure
      file are included only once (No #ifdef confusion).

Julia ⇔ C/C++: Module interface

  • C++ exposes interfaces using “public” .h/.hpp files whereas Julia modules export
    symbols that are intended for their users.
    • Often, Julia modules simply add functionality by generating new “methods” to existing
      functions (ex: Base.push!).
    • Developers of Julia packages therefore cannot rely on header files for interface
      documentation.
    • Interfaces for Julia packages are typically described using docstrings, README.md,
      static web pages, …
  • Some developers choose not to export all symbols required to use their package/module.
    • Users might be expected to access these components by qualifying functions/structs/…
      with the package/module name (ex: MyModule.run_this_task(...)).

Julia ⇔ C/C++: Quick reference

Software Concept Julia C/C++
unnamed scope begin … end { … }
function scope function x() … end int x() { … }
global scope module MyMod … end namespace MyNS { … }
software module A Julia “package” .h/.hpp files
+compiled somelib.a
assembling
software modules		 SomePkg.jl: …
import subfile1.jl
import subfile2.jl
…		 $(AR) *.o ⇒ somelib.a
import
software module		 import SomePkg #include <somelib>
+link in somelib.a
module library LOAD_PATH[], *Git repository,
**custom package registry		 more .h/.hpp files
+bigger compiled somebiglib.a

* The Julia package manager supports registering multiple packages from a single Git repository.

* This allows users to house a library of related packages in a single repository.

** Julia registries are primarily designed to provide versionning & distribution of packages.

** Custom package registries can be used to create a type of module library.

9 Likes