We had a training on testing and package development with Python and Matlab… It is so much easier to do that with Julia… We just need to spread the message… I need to prepare a lunch-lecture on Julia…
13 Likes
times this attitude. Keep moving forward, using and learning Julia. There are many amazing people in the community and wonderful things built with it. That spirit of building and reaching toward launch, as exemplified in this clip, is something rare and worth sustaining.
3 Likes
That’d be correct. The new idea of Julia is that if you’re skilled enough in what you’re doing, you can do almost anything theoretically possible. Like, if some drone startup wants to optimize the drone by differentiating through finite element simulation and then use the sparse hessian to optimize for something, well, that could be something Julia excel, Likely, this would be something very few tools can do in practice, and Julia could get to be one of them.
Julia’s unrivaled capability is its ability to let users connect the dots from seemingly disparate fields and bring them together to create something awesome. That would be where Julia reign supreme. If we can take a stand there, there is almost certainly no defeat.
However, as Steve Jobs said, you cannot connect the dots looking forward. That means you need to explore various applications in the Julia ecosystem and even invite diverse areas of compute to develop in Julia.
I’d say keep inviting new people from different fields. No need for a lot of people. If we can get 2% of the most enthusiastic devs who can then put in 20% of the work to create a package that’s 80% as good as the best tool, but that happens to be able to work well with other tools in the Julia ecosystem, do that.
It looks like we’re in a long game.
1 Like
I’m a relative newbe and started using julia only a few months ago. My previous programming experience was from the early '90s and earlier in geoscience, using Fortran and SAS for straightforward data analysis. I have no experience of other, modern languages.
I was initially attracted to julia by its promise of speed and capability for data analysis but also because it promised to be good for general purpose computing, too. I have been blown away by what I have been able to do since I started. It has been a steep learning curve, but amazingly productive.
Every time I come to this community for support, I get what I need from a friendly and helpful community of real experts.
However, and this is a ridiculously small “but”, the clear emphasis here on numerical methods over-awes me every time. I know this is a particular focus for julia and its user community, but anyone interested in “general computing” may be intimidated by what they find here.
I’m not for a minute suggesting it is bad that julia’s strengths are reflected in this discourse. That is absolutely how it should be. But, like @mnemnion, I’d welcome more general computing presence, too.
I’m not really able to offer a solution to this, but do think the focus on numerical computing might become a self-reinforcing reality and it really shouldn’t.
I can’t imagine switching away from julia - it’s too effective and I enjoy using it too much. I promote it to colleagues who, like me, are never going to be “hard core” coders. They may occasionally dabble in python. Postgraduate level numerical computing won’t ever be their thing, but julia could and should be!
19 Likes
It’s super unfortunate if this is the effect such sentiment has on you. I no longer spend much energy bemoaning the size of the Julia user base - I think I’ve been convinced by previous discussion that it’s not a problem - but in the past I did, and if that bummed you out, I’m sorry.
In my own mind, the motivation for the sentiment was that this is an amazing thing that you and so many others have built, and I just wish that more people could capitalize on that goodness, that it got more of the support it deserves. All of that is still true!
14 Likes
The person Stefan was responding to has a tendency to post very broad open ended questions bemoaning difficulties and unfairness and I don’t think it was targeted towards you at all.
9 Likes
Oh yeah, I know that it wasn’t. But I’m implicated, because I think that I’ve made similar comments in the past without a full consideration of the effects that those comments would have on the people putting in so much effort towards making this language amazing.
I hope that explaining where I was coming from might offer a different framing, and hopefully reduce the sting. I think it’s inevitable that some people, especially those newly excited about the language, will look around at the fact that it’s not more widely adopted and bemoan that lack - it’s just in the nature of fandom. But I think this can and should be seen in the light of
Wow, this thing is great, I wish everyone could see that
!
rather than
Ugh, it sucks that its so hard to get more people involved
Even if the later sentiment is what’s explicitly staid 
14 Likes
I know I’m chiming in a bit late on this discussion, but I would really want to emphasize the importance of the following comment
The state of Julia, to me, is a lot like the state of Industry 4.0 and digital twins. It’s full of promise, with people working on it, but not very many people are using it. Nevertheless, people who use it well are going to dominate. Moreover, Scientific ML is going to be the tool that makes digital twins actually useful.
The main problem is the fact that people unfamiliar with this area think that deep learning is enough to solve industrial problems. However, the data is generally not informative enough to fully define your operation. A real Industry 4.0 solution requires the seamless integration of expert knowledge and data to achieve something that is greater than the sum of its parts. But this seamless integration requires us to enable experts to easily build performant code that can be automatically differentiated, and iterated over through differential equation systems that combine first principles and black box models. The work that @ChrisRackauckas and his team have done in this area is nothing short of amazing. Julia is pretty much there (Enzyme.jl will really help) and is miles ahead of anyone else in this area. The problem is that all the chatter around AI obfuscates a few important points that people need to be aware of:
- Deep learning isn’t enough to make Industry 4.0 useful. The datasets require expert knowledge to make sense of, which means you really need to take a Scientific Machine Learning (SciML) approach.
- Feature engineering isn’t SciML; it can help but it’s extremely noisy; SciML can solve the problem more thoroughly/reliably by cutting through the noise.
- Physics-Informed ML is not SciML. Training a neural network on a bunch of simulations is a step forward, but the large number of unknown parameters still makes the approach quite brittle, and it’s very difficult to expand models to include more changing unknowns. SciML allows you to break the black box model components down into smaller, more tractable problems.
- SciML is more difficult, because you need people with broad technical knowledge to build these models, so they’re hard to scale. However, a single model of an industrial asset can provide a lot of economic value, so it is worth building properly.
- Julia has a SciML ecosystem that is far and beyond any other ecosystem out there.
The overall idea of SciML is not new: build high-level models with first principles and using small machine learning models to fill in unknowns. This is done all the time in thermodynamics, but has historically been about getting data from carefully controlled experiments to make it easy to “train” these models with high fidelity data free from confounding variables. The technological advancement is the ability to sufficiently train models like these as a deeply nested problem inside a large model with routine data.
7 Likes
Thank you for sharing your wonderful findings. I am trying to do similar things to your research. That is how many people are downloading for a given Julia package.
I mean I would like to get data something like below:
package_uuid, year, request_count
<UUIDExample>, 2020, 10
<UUIDExample>, 2021, 130
<UUIDExample>, 2022, 119
<UUIDExample>, 2022, 1120
I know there is a post Announcing: package download stats! that allows us to investigate package download statistics by myself. Can I get package download statistics “by year” or “by month” instead of “by date”? As far as I know we can get information from the date field in a CSV file named package_requests_by_date, which only covers from 1 month ago to today.