# Planning Large Projects

**URL:** <https://discourse.julialang.org/t/planning-large-projects/28562>\
**Category:** Tooling\
**Created:** [September 9, 2019, 2:17pm UTC](https://discourse.julialang.org/t/planning-large-projects/28562 "2019-09-09T14:17:57Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![anon92994695](https://avatars.discourse-cdn.com/v4/letter/a/ce7236/32.png) [@anon92994695](https://discourse.julialang.org/u/anon92994695)\
**Post date:** [September 9, 2019, 2:17pm UTC](https://discourse.julialang.org/t/planning-large-projects/28562/1 "2019-09-09T14:17:57Z")

</div>

Hi Everyone,

For the past two weeks or so I have been working on a large project. Potentially, very large? What tools do you all use as solo developers to keep the planning of projects like these straight? Is there a Project to UML type tool for julia packages so I can flowchart where the heck everything goes and if current placement of functionality makes sense. I just spent a half an hour copy pasting code and shuffling things to get my export statements to compile (yikes).

Any advice even slightly off-topic appreciated

---

<div class="post-metadata">

**Author:** ![Tamas\_Papp](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tamas_papp/32/25949_2.png) [@Tamas\_Papp](https://discourse.julialang.org/u/Tamas_Papp)\
**Post date:** [September 9, 2019, 2:55pm UTC](https://discourse.julialang.org/t/planning-large-projects/28562/2 "2019-09-09T14:55:30Z")

</div>

> [@anon92994695](#):
>
> Potentially, very large?

That’s a very subjective term, we don’t know what that means for you. Very powerful Julia packages can be written in 5-10k LOC.

> [@anon92994695](#):
>
> I just spent a half an hour copy pasting code and shuffling things to get my export statements to compile (yikes).

Not sure what you mean here,

```julia
export foo

```

involves no compilation.

In any case, I would just get prototyping, and hammer out an interface, benchmark, refine, and repeat until I like it. Languages which support quick prototyping support this workflow rather well.

---

<div class="post-metadata">

**Author:** ![anon92994695](https://avatars.discourse-cdn.com/v4/letter/a/ce7236/32.png) [@anon92994695](https://discourse.julialang.org/u/anon92994695)\
**Post date:** [September 9, 2019, 3:07pm UTC](https://discourse.julialang.org/t/planning-large-projects/28562/3 "2019-09-09T15:07:45Z")

</div>

@Tamas_Papp -  
This may lead to being around 10k+ lines(without tests and documentation) if I can’t find packages to support some of the efforts in it. I’m at ~2k lines(no comments/doc strings) and probably 1/10th the functionality I need/want is in place. Large as in, there is a lot of functionality that interacts with a lot of other functions.

What I meant by shuffling exports was, I needed to break functionality across files. To do so I had to also organize constructor functions which depended on other structs to be defined in appropriate order so they existed. Sorry if that wasn’t clear.

My current approach involves a small white board and thinking very hard. I may end up breaking up the project into a base package and other small components, but I was hoping there were organizational tools to help me out as well.

---

<div class="post-metadata">

**Author:** ![anon92994695](https://avatars.discourse-cdn.com/v4/letter/a/ce7236/32.png) [@anon92994695](https://discourse.julialang.org/u/anon92994695)\
**Post date:** [September 9, 2019, 3:17pm UTC](https://discourse.julialang.org/t/planning-large-projects/28562/4 "2019-09-09T15:17:42Z")

</div>

How hard would it be to make an Atom/Juno plug-in to sprawl a Julia code-base and make an interactive UML like chart of it?

---

<div class="post-metadata">

**Author:** ![Tamas\_Papp](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tamas_papp/32/25949_2.png) [@Tamas\_Papp](https://discourse.julialang.org/u/Tamas_Papp)\
**Post date:** [September 9, 2019, 3:50pm UTC](https://discourse.julialang.org/t/planning-large-projects/28562/5 "2019-09-09T15:50:07Z")

</div>

I think you are on the wrong track here. Julia is its own best prototyping language. Most of this classic essay applies, replacing Lisp with Julia (which is also a Lisp at heart):

[http://www.nhplace.com/kent/PS/Hindsight.html](http://www.nhplace.com/kent/PS/Hindsight.html)

Effectively, that’s another two-language problem solved. 😉

---

<div class="post-metadata">

**Author:** ![anon92994695](https://avatars.discourse-cdn.com/v4/letter/a/ce7236/32.png) [@anon92994695](https://discourse.julialang.org/u/anon92994695)\
**Post date:** [September 9, 2019, 3:56pm UTC](https://discourse.julialang.org/t/planning-large-projects/28562/6 "2019-09-09T15:56:51Z")

</div>

I’m not trying to prototype my project. I’m trying to visualize it? Maybe I’m not being clear enough.

I’d like to map out my project in something like a UML flowchart: [What is Unified Modeling Language (UML)?](https://www.visual-paradigm.com/guide/uml-unified-modeling-language/what-is-uml/)  
To help me keep track of everything. Are there tools for this?

---

<div class="post-metadata">

**Author:** ![Tamas\_Papp](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tamas_papp/32/25949_2.png) [@Tamas\_Papp](https://discourse.julialang.org/u/Tamas_Papp)\
**Post date:** [September 9, 2019, 4:00pm UTC](https://discourse.julialang.org/t/planning-large-projects/28562/7 "2019-09-09T16:00:48Z")

</div>

Not that I am aware of.

But note that in Julia elaborate towers of type hierarchies are not as important, useful, or encouraged as in C++ and friends.

So I would say that if you need a tool like this, you are writing heavily non-idiomatic Julia code.

---

<div class="post-metadata">

**Author:** ![anon92994695](https://avatars.discourse-cdn.com/v4/letter/a/ce7236/32.png) [@anon92994695](https://discourse.julialang.org/u/anon92994695)\
**Post date:** [September 9, 2019, 4:07pm UTC](https://discourse.julialang.org/t/planning-large-projects/28562/8 "2019-09-09T16:07:34Z")

</div>

I currently have no type hierarchies that extend beyond two “orders”. IE:

```julia
abstract type Fruit end

struct Apple <: Fruit
   color::String
end

```

If that’s heavily non-idiomatic julia code I don’t really know what to say…

Think this is the second time you’ve implied I was making bad design decisions without seeing my code? I am writing a physics package… There are, unsurprisingly a lot of things in physics that interact with one another and believe it or not multiple paradigms to view the same phenomena.

---

<div class="post-metadata">

**Author:** ![Tamas\_Papp](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tamas_papp/32/25949_2.png) [@Tamas\_Papp](https://discourse.julialang.org/u/Tamas_Papp)\
**Post date:** [September 9, 2019, 4:17pm UTC](https://discourse.julialang.org/t/planning-large-projects/28562/9 "2019-09-09T16:17:19Z")

</div>

> [@anon92994695](#):
>
> you’ve implied I was making bad design decisions without seeing my code?

Please don’t take this personally. I think that warning people about the dangers of some approach is generally useful.

I may of course be wrong (but since you did not share any actual code to start with, it is hard to be more specific). Just experiment with what you find useful and you will find what works best for you.

That said, if there is even a remote chance that you will be showing your code to others (asking for advice, cooperating), please consider following the

[https://docs.julialang.org/en/v1/manual/style-guide/](https://docs.julialang.org/en/v1/manual/style-guide/)

(type names capitalized etc)

---

<div class="post-metadata">

**Author:** ![anon92994695](https://avatars.discourse-cdn.com/v4/letter/a/ce7236/32.png) [@anon92994695](https://discourse.julialang.org/u/anon92994695)\
**Post date:** [September 9, 2019, 4:20pm UTC](https://discourse.julialang.org/t/planning-large-projects/28562/10 "2019-09-09T16:20:32Z")

</div>

I wrote that in like 10 seconds to respond… Sorry to bother your eyes I will amend my most grievous error…  
In my code I follow conventions set by the Julia community…  
…

---

<div class="post-metadata">

**Author:** ![jpsamaroo](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jpsamaroo/32/46804_2.png) [@jpsamaroo](https://discourse.julialang.org/u/jpsamaroo)\
**Post date:** [September 9, 2019, 5:48pm UTC](https://discourse.julialang.org/t/planning-large-projects/28562/11 "2019-09-09T17:48:18Z")

</div>

As @Tamas_Papp has stated, having some code (even just mock-ups) for us to look at would help us help you immensely. Of course, until we’ve seen some actual code, it wouldn’t be fair for us to judge said code as good _or_ bad; so the least we can do is give very broad, non-specific advice which often will come across as implying that your code is bad (even thought that’s hopefully not the intent) because of the nature of the advice itself (i.e. we can’t complement your excellent craftsmanship if we can’t see what you’re working on).

Anyway, I will play devil’s advocate and say that I’d be interested in some tooling which can render the structure of a package or project in some sort of graph-like or hierarchical manner. Some packages (or even organizations, like DiffEq) have a _lot_ of code and API surface, and it would be really cool to see it all laid out as a graph that connects the dots. It could also make large API refactoring efforts easier by showing at a glance everything downstream that needs changing when something upstream changes.

---

<div class="post-metadata">

**Author:** ![Mason](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mason/32/2423_2.png) [@Mason](https://discourse.julialang.org/u/Mason)\
**Post date:** [September 9, 2019, 6:31pm UTC](https://discourse.julialang.org/t/planning-large-projects/28562/12 "2019-09-09T18:31:16Z")

</div>

I think it could be an interesting project to build something with an interface kinda like [Cthulhu.jl](https://github.com/JuliaDebug/Cthulhu.jl) where instead of descending into `@code_typed`, it displays a type hierarchy like this [1]:

```julia
Number (Core)
	Complex (Base)
	Real (Core)
		AbstractFloat (Core)
			BigFloat (Base.MPFR)
			Float16 (Core)
			Float32 (Core)
			Float64 (Core)
		AbstractIrrational (Base)
			Irrational (Base)
		Integer (Core)
			Bool (Core)
			Signed (Core)
				BigInt (Base.GMP)
				Int128 (Core)
				Int16 (Core)
				Int32 (Core)
				Int64 (Core)
				Int8 (Core)
			Unsigned (Core)
				UInt128 (Core)
				UInt16 (Core)
				UInt32 (Core)
				UInt64 (Core)
				UInt8 (Core)
		Rational (Base)

```

and then the user can navigate through the hierarchy and see the output of `methodswith` on each type, or the names exported by each module.

Perhaps that sort of functionality would help the OP?

* * *

[1] This type tree is generated by

```julia
function showtree(T::Type, level=0)
    println("\t" ^ level, T, " ($(parentmodule(T)))")
    for t in subtypes(T)
        showtypetree(t, level+1)
    end
end

showtree(Number)

```

which I adapted from [Introducing Julia/Types - Wikibooks, open books for an open world](https://en.wikibooks.org/wiki/Introducing_Julia/Types).

---

<div class="post-metadata">

**Author:** ![anon92994695](https://avatars.discourse-cdn.com/v4/letter/a/ce7236/32.png) [@anon92994695](https://discourse.julialang.org/u/anon92994695)\
**Post date:** [September 9, 2019, 7:07pm UTC](https://discourse.julialang.org/t/planning-large-projects/28562/13 "2019-09-09T19:07:52Z")

</div>

Mason this is more what I’m talking about!

I would like to see how types are related, what functions accept certain types. Also what files those things are contained in from a project julia file (or multiple)?

I know in python there is a tool that does the following : [https://www.logilab.org/blogentry/6883](https://www.logilab.org/blogentry/6883)

I guess consider files as nodes in a graph? Or maybe have the option to consider types as nodes? I’m not sure the best way to lay out julia code like this, but something like this would be excellent for me.

It’s really hard to grok thousands of lines of code, even as the author sometimes, and if I weren’t the author I might be mortified. Moreso - I’d love to know if I’m doing anything stupid from a higher level - can I merge types to something more generic, do I have a lot of the “same” functionality that can be abstracted. Etc.

---

<div class="post-metadata">

**Author:** ![anon92994695](https://avatars.discourse-cdn.com/v4/letter/a/ce7236/32.png) [@anon92994695](https://discourse.julialang.org/u/anon92994695)\
**Post date:** [September 9, 2019, 7:15pm UTC](https://discourse.julialang.org/t/planning-large-projects/28562/14 "2019-09-09T19:15:40Z")

</div>

@jpsamaroo I’m not asking for a code review or compliments or anything. I’m asking for tooling, I’m sorry if this hasn’t been made clear. I agree though a graph structure could easily be utilized to make code visual once it has reached a certain kind of “critical mass”. That’s what I’m trying to inquire about. Are there tools like this written for Julia? I know there are for other languages.

---

<div class="post-metadata">

**Author:** ![tamasgal](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tamasgal/32/27946_2.png) [@tamasgal](https://discourse.julialang.org/u/tamasgal)\
**Post date:** [September 9, 2019, 7:28pm UTC](https://discourse.julialang.org/t/planning-large-projects/28562/15 "2019-09-09T19:28:16Z")

</div>

Sorry for being off-topic: just out of curiosity, since you mentioned it’s a physics package (and physics is free 😉 ), is there any reason why it’s closed source yet?

---

<div class="post-metadata">

**Author:** ![anon92994695](https://avatars.discourse-cdn.com/v4/letter/a/ce7236/32.png) [@anon92994695](https://discourse.julialang.org/u/anon92994695)\
**Post date:** [September 9, 2019, 7:31pm UTC](https://discourse.julialang.org/t/planning-large-projects/28562/16 "2019-09-09T19:31:00Z")

</div>

It’s closed source for one reason,

1. The useful functionality in it is all in islands and I am trying to unify it.

It will be MIT licensed once it’s worth making public. It’s a pretty large undertaking to even get the essence of what I want done… I speculate it may take 2 months or more. Right now I have a bit of free-time (inbetween jobs) but once I start the new job I won’t have nearly as much free-time.

---

<div class="post-metadata">

**Author:** ![tamasgal](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tamasgal/32/27946_2.png) [@tamasgal](https://discourse.julialang.org/u/tamasgal)\
**Post date:** [September 9, 2019, 7:38pm UTC](https://discourse.julialang.org/t/planning-large-projects/28562/17 "2019-09-09T19:38:50Z")

</div>

I really encourage you to make it public unless there are some good reasons to hide it (commercialising, licensing, NDA, younameit). I often see people being shy, hiding their stuff because they think it’s not “good/mature enough” or just have the feeling that they will be eaten alive.  
The open source community is tremendously helpful and people really prefer to help other people instead of blaming them for writing bad code or whatever. You can totally assume that most of your stuff is just ignored, but if it’s public, it’s much easier to get help.

That said, I think you could have had some really nice feedback and suggestions within the last hours instead of wild guesses what you are up to and dealing with and some critical comments that you probably do something fundamentally wrong.  
Not to mention that hiding such information often leads to things like the [XY problem](https://mywiki.wooledge.org/XyProblem) 😉

---

<div class="post-metadata">

**Author:** ![ericphanson](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ericphanson/32/215186_2.png) [@ericphanson](https://discourse.julialang.org/u/ericphanson)\
**Post date:** [September 9, 2019, 7:42pm UTC](https://discourse.julialang.org/t/planning-large-projects/28562/18 "2019-09-09T19:42:43Z")

</div>

By the way, [GraphRecipes.jl](https://github.com/JuliaPlots/GraphRecipes.jl) has some similar features for visualizing trees

---

<div class="post-metadata">

**Author:** ![anon92994695](https://avatars.discourse-cdn.com/v4/letter/a/ce7236/32.png) [@anon92994695](https://discourse.julialang.org/u/anon92994695)\
**Post date:** [September 9, 2019, 7:47pm UTC](https://discourse.julialang.org/t/planning-large-projects/28562/19 "2019-09-09T19:47:15Z")

</div>

No I really mean as it currently stands the package is very jumbled. The real core utility of my package is glueing between domains. Without it, it’s a pile of scripts(what I have now). Once I finish the glue I will happily share, just like I did with my other package([GitHub - caseykneale/ChemometricsTools.jl: A collection of tools for chemometrics and machine learning written in Julia.](https://github.com/caseykneale/ChemometricsTools.jl)). There’s no money to be made here, just trying to draw people into using Julia… Optics is my hobby.

Alls I’m asking about is code inspection tools. It seems there aren’t many and that’s A-okay. I’m starting to see some good ideas on how to build one.

---

<div class="post-metadata">

**Author:** ![anon92994695](https://avatars.discourse-cdn.com/v4/letter/a/ce7236/32.png) [@anon92994695](https://discourse.julialang.org/u/anon92994695)\
**Post date:** [September 9, 2019, 7:48pm UTC](https://discourse.julialang.org/t/planning-large-projects/28562/20 "2019-09-09T19:48:10Z")

</div>

This is a delicious find! Hmmm… I’m wondering if I could spin up a prototype really quickly here. In an ideal world - a tool like this could be interactive, but man I can deal.

[Next page](https://discourse.julialang.org/t/planning-large-projects/28562.md?page=2)