I keep circling around the same issue from a variety of different vantage points, and I think that having first-class support for local, unregistered packages (just local files, not repositories) would solve a lot of problems that I see in structuring Julia programs.
We have a repo that has our aircraft simulation in it. The current package manager does a great job of managing our dependence on external packages, like HDF5 and YAML. I love it.
We also have local modules for reusable functionality. One module might be, e.g., Aerodynamics. Another might be CommandTypes.
Sometimes, we’re making a huge simulation with tons of complex stuff in it, and we need Aerodynamics in there somewhere. Sometimes, we’re making a little tool to explore our aerodynamics, so it needs Aerodynamics too. Maybe Aerodynamics needs to know about common types as well, like AileronCommand, and lots of other things need to know about the AileronCommand.
If the AileronCommand is part of a local module, then somehow Aerodynamics needs to know where that module is loaded, or it needs to load it from its parent, and this creates weird scope issues. See the developing thread in Import types "above" a module with ..: for consistency and brevity?.
If the AileronCommand is part of a local, unregistered package, then everything that needs to know about it can just do
using CommandTypes, where
CommandTypes is available via
]dev path/to/CommandTypes. However, if Aerodynamics is also a local, unregistered package, then not only does it need to do
]dev CommonTypes (which is fine), but anyone that wants to use Aerodynamics also has to do
]dev CommonTypes. That is, they have to resolve all local dependencies manually. See "Unsatisfiable requirements" when local packages use local packages. It’s not the end of the world (I just spent all afternoon dealing with Python stuff that was at least this cumbersome), but it is strangely messy for Julia.
If we could just
]add path/to/LocalPackage, have that work like
]dev, and just have everything “agree” to use whatever’s in that local path rather than worrying about versions of things that have no versions because they don’t have their own git repos, then I think this would all just work. (I recognize that it’s likely that I’m missing something here, but it sure seems like this would all work, and there seem to be others have similar problems.)
Just to be super-duper clear about this: having tons of packages in different repos is a good workflow for some stuff, and this seems to be the Julian way currently, but it doesn’t make sense for a lot of other stuff. Some projects are going to have everything in one repo, especially when Julia isn’t the only language (Julia is a small minority in our code base for now), so in those cases, local, unregistered packages seems like a good way forward.
What do folks think?