If you’re interested in making an interactive way to generate and move through trees of types / functions (though not fancy plot graphics), my humble package ReplMaker.jl might be of assistance. It won’t be straightforward, but it should be easier than wrestling with Julia’s REPL code directly.
3 Likes
Cool! I’ll consider it. I may try for a super hacked down version first…
There is another tool I’ve been kicking around making but haven’t done as well. This may be a bit of a distraction from my main goal but it sounds like fun and maybe people here will take ownership of it and make it way better…
After sitting down for a half hour or so I came to a daunting realization that performing this project would require basically rewriting a way to read the julia call stack… Does anyone who knows the insides of Julia know if there’s a way to highjack how julia compiles code?
Have a look at Cassette.jl.
1 Like
Cool! Alright I have some reading to do…
I doubt you really need to go down to that level, Julia has pretty good introspection tools that should help you getting the information you need, e.g. if you want to list all the types in a module you can do something like this:
using Distributions
mod = Distributions
fields = [getfield(mod, n) for n in names(mod, all=false)]
types = filter(x -> typeof(x) <: DataType, fields)
stypes = supertype.(types)
10 Likes
Good to know, Yea for practical purposes this sort of detail should work just fine. I’ll get cracking on this when I get a chance too. Right now I’m in a place where I can’t really write code unfortunately. Maybe I’ll get a chance to hack away in a few days.
Edit - actually running that script on my own package found an export statement for a function that was deleted! Nice.
It sounds like you really don’t want a 10k package to glue between domains, but an interoperating ecosystem of small packages like DiffEq. Julia is the one of the best languages around for code sharing and reuse in this way. I’ve never actually written a project in a single packages, always multiple because the parts are always independently useful in separate domains, and different enough that they should be separate anyway. It makes them easier to grok individually, and forces a modular approach that is better in the long term anyway.
One other comment from looking at ChemometricsTools.jl: You say you’re following conventions but I can’t see it! Camel case function names and filenames, and lowercase type are totally foreign to most of us, and will be a barrier integration. The style guide demonstrates things fairly well.
6 Likes
some ppl do reply(?), not even reply, just put a comment on a question in a very weird way!! If I were “ckneale” for sure I would have regretted why I did post a question!!! be friendly and try to be helpful if you cannot, you do not need to answer or put any comment from top to bottom…
2 Likes
From where I’m standing it seems to me that @anon92994695 got pretty good feedback.
I’m sure the critiques that are posted above were all meant well.
We all appreciate contributors to the Julia ecosystem, and we are just trying to help a member who asked a question. Perhaps some of the answers are more subjective than others, but that comes with the territory: everyone probably has some ideas on how to understand a project, and they may well be different from those held by other people.
4 Likes
@Raf - It is possible I could break a lot of the functionality into several packages. Even still though - having tools to assess code across the packages would be very helpful in compartmentalizing things/ refactoring.
What I meant was in my current project I am following proper code conventions. When I first wrote ChemometricsTools.jl I didn’t know the conventions, I think that was back when 0.70 was the latest version of Julia. I haven’t had the time to fix it - none of my users have complained either. Please feel free to file an issue or submit a PR.
@Hani - I kind of agree. Every thread I’ve made on here recently has received a lot of traction. I can’t quite figure out if people are that picky, angry at me, or what. It’s generally not very inviting. “The open source community will help you” - yadda yadda yadda I’ve had to do everything myself from the ground up while getting knit picked by people who are paid to contribute to open source. This is a hobby for me. I feel like people could be nicer about some things, because I’m only in this to have fun and to help. I think that’s the riff - some people here are paid to maintain high code quality and take things very seriously or else they’ll get fired or whatever. While I’m just having fun hacking away, in my free-time, making tools I enjoy using.
That being said - I’ve dealt with FAR more toxic communities than this. I got a PhD - trust me this is nothing. But, if it ever got really bad I’d just go back to writing C++ for fun.
Anyways on topic - I may or may not have time to devote to making a tool like this. I think I’m just going to do my planning old school for now (pencil and paper) and maybe the Julia community will come up with a UML type tool. The idea is at least in this thread - buried beneath the derailment.
On topic now: I wonder if UML is the right tool for Julia? UML seems to be well suited to objects talking to each other, expressing relationships between types, describing type hierarchies. Julia is in my opinion different: the multiple dispatch, function-based approach makes type hierarchies almost secondary.
So if UML is not the perfect tool, what is? I’ll admit I don’t know.
3 Likes
Thats a great question. I’m not entirely sure myself.
High level restructuring seems very abstract to anyone but the author of a package without these kinds of visualizations or an extreme familiarity from the users. I know it would be useful for me to say ‘query’ a struct or something like that and see the functions which take it as input or return it as output. Furthermore it would be nice to see what other types contain a certain type. Visualization of either of these items would be really helpful for me, and could make nice design documents for passerby’s.
There’s a lot of ways to look at things. I’m not a Julia expert, and I know for some packages it’s pretty easy to grok everything because it’s very self contained/small (“smallest shareable unit”). But, there are larger projects and efforts which maybe defy this type of design or end up being many such smaller units that an end user needs to be familiar with to understand how to manipulate/contribute.
Perhaps your are referring to Fun One Liners?
Let me make a (well-meant) comment: responses in a thread (to some extent) reflect the preciseness and clearness of the particular matter. That is to say that if a thread is started on a vague statement/question answers will almost inevitably be diffuse as well.
In any case, I hope you do not feel discouraged by the responses. Although it might not always look like it, people here generally have good intentions.
3 Likes
That’s not good - we want this to be a welcoming community. Certainly, some people can come off as a bit surly, but I’m really confident that everyone means well. I’m really sorry that it’s left you feeling unwelcome.
I think you have the wrong impression here - most of us are not paid for this. A lot of people (myself included) are academics that develop in julia for their research, but speaking for myself, my participation in this community is strictly off the clock.
I think I speak for many here when I say that “not as toxic as some open source communities” is a far lower bar than we want for this one. Keep calling stuff out.
13 Likes
I know exactly what you mean. If you’ve read sufficiently many posts on this forum then you start to learn whose comments you need to renormalise. But for a relative newcomer it can be very off-putting.
One thing I’d say though is that some of the „grumpiest“ posters are also amongst the most helpful especially with technicalities.
10 Likes
Since julia is homoiconic, it should be fairly straightforward to use reflection to generate UML.
The main problem is that Julia’s multi-dispatch leads to design patterns that escape traditional UML modeling – i.e. each conversion or promotion call contributes to a web of functor-like morphisms that allow multi-methods to match many more objects/classes than most other languages would allow. This leads to weird UML. Try doing it on paper for a package and see how it’s different than a similar library in C++ or Java.
2 Likes
So a while ago I started working on a tool for this:GitHub - caseykneale/Sherlock.jl: A high functioning package detective.
The idea is instead of a UML, just make it a graph plot like someone else suggested. That’d be enough to help out. Although that tool isn’t 'complete" it did help me organize and find cruft in a smaller project(maybe 4-5 src code files).
The planning aspect is, you could interactively, maybe by using Revise? map out a project with a bunch of empty structs, types, functions, etc. Why didn’t it get to that point you ask? …
I lost steam with it, the UI/UX was just too slow, not interactive, and the plotting algorithms needed help. At the time LG was in pretty heavy flux, and I believe the main author has since decided to leave the community :(. UI/UX for Julia is in the same state - roughly speaking.
2 Likes