I’m an instructor teaching in a Data Science program, and have recently discovered a bit of an… odd situation related to the for-profit side of Julia.
I mostly teach Python (it’s the first language of data science at the moment, so I think to do otherwise would be unfair to students who want jobs), but am pulling in more and more Julia.
We’re also pushing more and more cloud computing, so I wanted to add a “Julia in the cloud” section to my courses. But while there are great tutorials and open source tools for launching dask on cloud platforms that are really easy to use, no such thing seems to exist for Julia. If you’re on a static cluster there’s ClusterManager.jl, but nothing for kubernetes or cloud APIs.
Except… for this: JuliaRun, the closed-source tools Julia Computing is selling.
I’m super on board with Julia Computing making money by providing support to enterprises, and making adaptations to Julia to make it more accessible. Indeed, Dask now has something similar. But hiding such a critical feature behind a paywall feels very problematic, and has the potential to really hinder Julia adoption.
I don’t understand what you mean, you are free to develop an open-source alternative to JuliaRun if you need it, or even start your own company and sell such a product.
Fair. I guess its more about the tension of a lot of core developers building private forks of Julia, which then reduces the likelihood of contributing to the open source community.
Part of this, I suppose, is that I think of part of Julia Computing’s goal as being to promote the growth of Julia and support core devs financially in doing so. The way it was setup and pitched, as I understand it, is to add value in the domains of support for enterprise, not by creating private libraries, something that has clear benefit to the community. But here we have a bit of a re-direction of efforts away from open source.
Can’t follow you. Why is Julia Computing obligated to develop everything as open source? Just because it would have some benefit to the community?
Julia Computing has some kind of business plan, which is build on top of Julias open source environment. As these are mainly those people who gave us Julia at first I really can’t see any obligations (and in my opinion even not if these were just other people benefitting from Julia as open source, that is the nature of open source and everything you can do and not is described in the license). If they need private libraries for their business to prosper this is their right to do so.
I can’t see or imagine any tension in principle.
I don’t really think there’s a “redirection”-- if JuliaComputing didn’t exist, they wouldn’t be there to hire people to make JuliaRun, so it’s not like the developers of JuliaRun would automatically be doing the same thing but open source.
All fair. And you’re right, its possible no one would have developed in the open space domain.
I guess my sense was just that Julia Computing was aiming to develop a business model that didn’t create a tension between what’s open and what’s not. A lot of what they do doesn’t create such a tension – helping businesses figure out how to deal with their own idiosyncratic needs, accepting funding to focus on developing something that does go into the core language, In person training, help trouble shooting unusual datasets, etc.
And you’re also right that companies can making $$ under the license. I think things feel just a little different when it’s the BDFL who has a private library that could be really helpful to the broader community that’s private.
Open source runs on people contributing things because they feel the community is oriented towards openness and the public good. Having the core devs building private libraries that are likely of great public interest feels like it creates a tension with that community ethos. Or at least feels like it to me, though maybe I’m alone in that!
That’s for the pointers to JuliaCloud! Very helpful. My university has a strong Azure relationship so I’m not sure I can use it, but I can point students to it.
I appreciate they’re under different umbrellas, but Julia Computing includes the four creators of Julia + Keno + Edelman + etc. etc., who are also major participants in… well, everything Julia? The fact that they wear multiple hats, and those different hats may have different incentives if JuliaComputing is moving into the domain of selling private packages is why I think it’s relevant that this is a discussion about “Julia Computing” and not “Joe’s Julia Packages.”
EDIT: As that site itself notes: “The company employs many of the top contributors to Julia itself (half of the top ten contributors by commits) and to major Julia packages, especially for data science and machine learning.”
So I’m not sure this distinction is as real as it looks on paper.
Fair – much is self-interested. But I definitely don’t think all. I really doubt the Queryverse was developed because @davidanthoff thought it’d help him get tenure, and I’m pretty sure @bkamins didn’t have to essentially take over the development of DataFrames because otherwise he couldn’t do his economics research…
Yes, may be it is sometimes more altruistic than the opposite. But we do not know how many are from those and how much each one profits for themself. That’s very much speculating.
The point is, based on this speculation assuming some tension, is just constructing tension where nothing may exist.
Another point: I believe that Julia is very dependent on the community efforts. Julia without the whole package environment would still be notable, for me it would still be a great and very usable achievement, but building business on just plain Julia? I don’t know if this would work out well. So I truly believe that the people at Julia Computing do have a great interest in us, the community. And because I believe that, I am happy if they have a good and prosper living with their Julia Computing earnings.
There will be edge cases, but again, I would say, it is on them to decide and on us to take it as it is.
We are (I am) in no way dependent on Julia. If it doesn’t fit anymore I will happily move on. (But it doesn’t look so for quite some future I imagine).
Except that I ended up here because there’s this great library that’s not public, so… not sure this falls under “just constructing tension for tension’s sake”…
I think what you are looking for is easy ways to launch Julia on clusters. The ClusterManagers.jl projects in JuliaParallel and the cloud APIs in JuliaCloud are what probably want. In fact, there is also an open source Kubernetes client - Kuber.jl, which has been open sourced by Julia Computing. The ease of launching Julia on various kinds of clusters needs to improve - there is no doubt about that.
But JuliaRun is not a library for launching clusters in a variety of different cluster environments. JuliaRun is a cloud platform for companies to run Julia behind a firewall, scale jobs with Kubernetes, with authenticated user accounts and various enterprise features. We continue to work towards making these capabilities easy for individual Julia users to access - and we had an early demo of this at JuliaCon.
As many in this thread have pointed out, Julia Computing releases lots of open source software, and also builds many products commercially for our customers and users.
Moreover, the vast majority of development work that @jeff.bezanson, @viralbshah, @mit.edelman and myself do is open source. Other people at Julia Computing tend to work on the commercial products, and we oversee that work, but when we do programming, it is still almost all open source.
Thank you for the clarification. I think the fact that it is described as being “for scalable deployment of Julia in production for large-scale parallel simulations in the public or private cloud, whether it is AWS, Microsoft Azure, or Google Cloud.” made it seem like a resource for launching cloud instances on these services, as was the fact I was pointed to JuliaRun when I asked about trying to manage kubernetes in the ClusterManager.jl repo.
Kuber.jl sounds great, and I"ll check it out, thank you.
That’s great, and your contributions are all very much appreciated, believe me, and as noted above, I’m very grateful that Julia has managed to find opportunities to develop complimentary for-profit ventures to help support the community. I think there is an inescapable possible tension between profit / non-profit interests, and so I wanted to discuss it. From what @viralbshah says, sounds like there’s no issue here though.
You can take a look our recent packages: AzManagers, AzSessions and AzStorage. It definitely is not a turn key solution, and the documentation is still a bit lacking; but if you are working on Azure, then these might be a reasonable starting point for Azure scale-sets, storage and authentication.