For the past two weeks or so I have been working on a large project. Potentially, very large? What tools do you all use as solo developers to keep the planning of projects like these straight? Is there a Project to UML type tool for julia packages so I can flowchart where the heck everything goes and if current placement of functionality makes sense. I just spent a half an hour copy pasting code and shuffling things to get my export statements to compile (yikes).
That’s a very subjective term, we don’t know what that means for you. Very powerful Julia packages can be written in 5-10k LOC.
Not sure what you mean here,
export foo
involves no compilation.
In any case, I would just get prototyping, and hammer out an interface, benchmark, refine, and repeat until I like it. Languages which support quick prototyping support this workflow rather well.
@Tamas_Papp -
This may lead to being around 10k+ lines(without tests and documentation) if I can’t find packages to support some of the efforts in it. I’m at ~2k lines(no comments/doc strings) and probably 1/10th the functionality I need/want is in place. Large as in, there is a lot of functionality that interacts with a lot of other functions.
What I meant by shuffling exports was, I needed to break functionality across files. To do so I had to also organize constructor functions which depended on other structs to be defined in appropriate order so they existed. Sorry if that wasn’t clear.
My current approach involves a small white board and thinking very hard. I may end up breaking up the project into a base package and other small components, but I was hoping there were organizational tools to help me out as well.
I think you are on the wrong track here. Julia is its own best prototyping language. Most of this classic essay applies, replacing Lisp with Julia (which is also a Lisp at heart):
I’m not trying to prototype my project. I’m trying to visualize it? Maybe I’m not being clear enough.
I’d like to map out my project in something like a UML flowchart: What is Unified Modeling Language (UML)?
To help me keep track of everything. Are there tools for this?
I currently have no type hierarchies that extend beyond two “orders”. IE:
abstract type Fruit end
struct Apple <: Fruit
color::String
end
If that’s heavily non-idiomatic julia code I don’t really know what to say…
Think this is the second time you’ve implied I was making bad design decisions without seeing my code? I am writing a physics package… There are, unsurprisingly a lot of things in physics that interact with one another and believe it or not multiple paradigms to view the same phenomena.
Please don’t take this personally. I think that warning people about the dangers of some approach is generally useful.
I may of course be wrong (but since you did not share any actual code to start with, it is hard to be more specific). Just experiment with what you find useful and you will find what works best for you.
That said, if there is even a remote chance that you will be showing your code to others (asking for advice, cooperating), please consider following the
I wrote that in like 10 seconds to respond… Sorry to bother your eyes I will amend my most grievous error…
In my code I follow conventions set by the Julia community…
…
As @Tamas_Papp has stated, having some code (even just mock-ups) for us to look at would help us help you immensely. Of course, until we’ve seen some actual code, it wouldn’t be fair for us to judge said code as good or bad; so the least we can do is give very broad, non-specific advice which often will come across as implying that your code is bad (even thought that’s hopefully not the intent) because of the nature of the advice itself (i.e. we can’t complement your excellent craftsmanship if we can’t see what you’re working on).
Anyway, I will play devil’s advocate and say that I’d be interested in some tooling which can render the structure of a package or project in some sort of graph-like or hierarchical manner. Some packages (or even organizations, like DiffEq) have a lot of code and API surface, and it would be really cool to see it all laid out as a graph that connects the dots. It could also make large API refactoring efforts easier by showing at a glance everything downstream that needs changing when something upstream changes.
I think it could be an interesting project to build something with an interface kinda like Cthulhu.jl where instead of descending into @code_typed, it displays a type hierarchy like this [1]:
and then the user can navigate through the hierarchy and see the output of methodswith on each type, or the names exported by each module.
Perhaps that sort of functionality would help the OP?
[1] This type tree is generated by
function showtree(T::Type, level=0)
println("\t" ^ level, T, " ($(parentmodule(T)))")
for t in subtypes(T)
showtypetree(t, level+1)
end
end
showtree(Number)
I would like to see how types are related, what functions accept certain types. Also what files those things are contained in from a project julia file (or multiple)?
I guess consider files as nodes in a graph? Or maybe have the option to consider types as nodes? I’m not sure the best way to lay out julia code like this, but something like this would be excellent for me.
It’s really hard to grok thousands of lines of code, even as the author sometimes, and if I weren’t the author I might be mortified. Moreso - I’d love to know if I’m doing anything stupid from a higher level - can I merge types to something more generic, do I have a lot of the “same” functionality that can be abstracted. Etc.
@jpsamaroo I’m not asking for a code review or compliments or anything. I’m asking for tooling, I’m sorry if this hasn’t been made clear. I agree though a graph structure could easily be utilized to make code visual once it has reached a certain kind of “critical mass”. That’s what I’m trying to inquire about. Are there tools like this written for Julia? I know there are for other languages.
Sorry for being off-topic: just out of curiosity, since you mentioned it’s a physics package (and physics is free ), is there any reason why it’s closed source yet?
The useful functionality in it is all in islands and I am trying to unify it.
It will be MIT licensed once it’s worth making public. It’s a pretty large undertaking to even get the essence of what I want done… I speculate it may take 2 months or more. Right now I have a bit of free-time (inbetween jobs) but once I start the new job I won’t have nearly as much free-time.
I really encourage you to make it public unless there are some good reasons to hide it (commercialising, licensing, NDA, younameit). I often see people being shy, hiding their stuff because they think it’s not “good/mature enough” or just have the feeling that they will be eaten alive.
The open source community is tremendously helpful and people really prefer to help other people instead of blaming them for writing bad code or whatever. You can totally assume that most of your stuff is just ignored, but if it’s public, it’s much easier to get help.
That said, I think you could have had some really nice feedback and suggestions within the last hours instead of wild guesses what you are up to and dealing with and some critical comments that you probably do something fundamentally wrong.
Not to mention that hiding such information often leads to things like the XY problem
No I really mean as it currently stands the package is very jumbled. The real core utility of my package is glueing between domains. Without it, it’s a pile of scripts(what I have now). Once I finish the glue I will happily share, just like I did with my other package(GitHub - caseykneale/ChemometricsTools.jl: A collection of tools for chemometrics and machine learning written in Julia.). There’s no money to be made here, just trying to draw people into using Julia… Optics is my hobby.
Alls I’m asking about is code inspection tools. It seems there aren’t many and that’s A-okay. I’m starting to see some good ideas on how to build one.
This is a delicious find! Hmmm… I’m wondering if I could spin up a prototype really quickly here. In an ideal world - a tool like this could be interactive, but man I can deal.
