JuliaComputing profit / non-profit tensions?

[nickeubank]

EXCELLENT! Thank you!!

[avik]

If access to Azure you were looking at, take a look https://github.com/JuliaComputing/Azure.jl, which as it happens was written by Julia Computing and open sourced.

[Tamas_Papp]

@nickeubank

Except that I ended up here because there’s this great library that’s not public, so… not sure this falls under “just constructing tension for tension’s sake”…

To be fair, it also sounds like complaining about a product that you need not being freely available.

I feel you pain. I am currently eyeing a very nice custom-built bike with a Rohloff hub and a belt drive. In hot-rod red. When I asked for a quote, I learned that they are asking for a lot of money instead of giving it to me free just because we both bikes.

[nickeubank]

@Tamas_Papp

To be fair, it also sounds like complaining about a product that you need not being freely available.

I apologize if it came across that way. FWIW, I promise this really is just an inquiry about how the Julia community operates with a semi-for-profit arm. I don’t actually have a need to put Julia on an Azure cluster myself – I’m fortunate enough to have a great faculty allocation on my university’s SLURM-managed cluster, for which ClusterManager is more than sufficient.

But I’m an instructor in a Masters in Data Science program, and while our program is primarily Python-oriented (as that is still the lingua franca of data science in industry, and what we feel our students should learn first), since I arrived here I’ve been pushing hard to get more Julia into the curriculum. As our students are mostly private-sector bound, and deeply interested in the cloud given it’s where many of the jobs are, their interest in Julia seems to depend in large part on the ability to use the language on a service like Azure or AWS.

Moreover, I design my classes to be free-standing, public, and free, so there’s no point in including things I don’t think most visitors have access too. This all came up because I wanted to replicate materials I’d written for Python in the cloud for Julia to entice more students to Julia, but couldn’t find a way to do it.

EDIT: typos.

EDIT 2: And thank you to people who’ve pointed out other resources. I’d stumbled across some of those (Azure.jl, for example, is super helpful for storage, but doesn’t seem to help with clusters, unless I’m missing something?), but definitely hadn’t seen these packages by @samtkaplan. I also promise I was digging into this before raising this question about JuliaRun, even if I wasn’t able to find everything people have posted here – I read older issues, asked about open-source tools here a week ago, and went over to ClusterManagers (who just pointed me to JuliaRun).

[kristoffer.carlsson]

@nickeubank

But I’m an instructor in a Masters in Data Science program, and while our program is primarily Python-oriented (as that is still the lingua franca of data science in industry, and what we feel our students should learn first),

Did you know that the creator of Python worked for Dropbox for many years. And that Dropbox has products you can actually pay for. Did you take that into consideration when you decided to use Python in your Data Science program? Was it a major source of worry?

[Tamas_Papp]

@nickeubank

how the Julia community operates with a semi-for-profit arm

You may be assuming more coordination about this than what actually happens.

Some members of the community make money from selling Julia software/services, but they don’t coordinate with the community at large, and there is no need to. It’s their business, in all senses of the word.

Personally, I don’t think there is any problem with this, in fact some for-profit activity can be part of a healthy software ecosystem.

Finally, my understanding is that JuliaRun is for users who run large-scale parallel computations. If they are doing this as part of they job, they should be OK to pay for this. If they aren’t, well, then they already have a very expensive hobby.

For your specific use case, instead of creating controversy where there isn’t one, I would have just approached JuliaComputing about using their tools in teaching. Sometimes that works — I can’t speak for them, but most companies have some formal or informal academic offering, it is good business sense.

[nickeubank]

@Tamas_Papp

Personally, I don’t think there is any problem with this, in fact some for-profit activity can be part of a healthy software ecosystem.

That seems to be the consensus view, which is great! There’s a reason I posted my subject with a question mark, and finished my post with a question – I was sincerely interested in people’s view of this.

But I don’t think it’s unreasonable to ask about this interface. I apologize if at any point my language was inappropriate, but open-source governance is tricky, and this seems a situation worth discussion, no?

@Tamas_Papp

Finally, my understanding is that JuliaRun is for users who run large-scale parallel computations. If they are doing this as part of they job, they should be OK to pay for this. If they aren’t, well, then they already have a very expensive hobby.

@Tamas_Papp

For your specific use case, instead of creating controversy where there isn’t one, I would have just approached JuliaComputing about using their tools in teaching. Sometimes that works — I can’t speak for them, but most companies have some formal or informal academic offering, it is good business sense.

For your specific use case, instead of creating controversy where there isn’t one, I would have just approached JuliaComputing about using their tools in teaching. Sometimes that works — I can’t speak for them, but most companies have some formal or informal academic offering, it is good business sense.

I imagine they may indeed have been willing to do so, but I think the concern, again, is that we’re talking about a resource not available to everyone (e.g. the student using materials online to develop skills who doesn’t have a job yet, or students at other programs whose instructors aren’t as motivated to advocate for Julia and jump through those hoops). Again, my inquiry is less about myself and more about Julia and accessibility of the language and platform.

[nickeubank]

@kristoffer.carlsson

Did you know that the creator of Python worked for Dropbox for many years. And that Dropbox has products you can actually pay for. Did you take that into consideration when you decided to use Python in your Data Science program? Was it a major source of worry?

A fair question! Yes, I know GVR was at Dropbox for a long time, and more importantly for a lot of the type of data science I do, that Wes McKinney has jumped around from one tech company to another. And as I noted above, dask now has a for-profit consulting arm.

But I’d never run into a situation where I couldn’t find an open source solution to a thing of general interest in Python land, in part (I suspect) because it’s built on a MUCH bigger pool of contributors.

(Also: don’t really have a choice – Python is where most of the jobs are. Julia is more of an elective I have to sell to the students…)

[oheil]

@nickeubank

But I’d never run into a situation where I couldn’t find an open source solution to a thing of general interest in Python land, in part (I suspect) because it’s built on a MUCH bigger pool of contributors.

There was just more time and therefor, over time, more contributions. Julia is still young.

[kristoffer.carlsson]

@nickeubank

But I’d never run into a situation where I couldn’t find an open source solution to a thing of general interest in Python land, in part (I suspect) because it’s built on a MUCH bigger pool of contributors.

So you are saying that until someone makes an open-source version of the features of JuliaRun it is “very problematic” that Julia Computing develops such a tool? I don’t really understand why? Because it would be better for you if that was available for free? Perhaps the income from such a tool allows many of the people working on Julia itself to put food on the table and do a lot of other good open source work that you can use?

[johnmyleswhite]

This thread seems to be getting tense with unclear benefits. It seems the summary is that:

• Some people perceive there to be a tension having Julia’s founders involved in a for-profit business.
• Others do not.

I’d say that it’s unlikely this will be resolved in a useful way. To those who don’t see a tension, implying a tension exists seems like FUD. To those who see a tension, implying a tension does not exist seems implausible.

Whether Julia’s current popularity is sufficient for teaching courses seems like an orthogonal issue that could get a separate thread unrelated to for-profit businesses.

[oheil]

More hearts given for tension relief

[nickeubank]

Excellent call.

[DNF]

It seems to me like @nickeubank asked an honest question, and was more-or-less satisfied with the responses.

After reading this (and previous somewhat similar threads) I personally don’t see any particular issues with how JuliaComputing operates, but I’m surprised to see the defensive tone in some of the responses here.

It certainly makes me think a few extra times before asking questions unless I can be sure that they are uncontroversial.

[oheil]

@DNF

It certainly makes me think a few extra times before asking questions unless I can be sure that they are uncontroversial.

Don’t put a burden on you like this. All in all controversial discussions are still good discussions. For me all went well here in this discussion.

@kristoffer.carlsson

That’s probably a good idea in general.

Uh oh, again controversial… more hearts…

[Tamas_Papp]

@DNF

I’m surprised to see the defensive tone in some of the responses here.

I am not sure that the tone of various responders is really defensive… just somewhat irritated and/or confused at the wording and implications of the the original post.

I think it is perfectly fine to ask controversial questions, but it is best to be explicit about them, instead of vague ellipses like

@nickeubank

a bit of an… odd situation

or claims that Julia Computing is

@nickeubank

hiding such a critical feature

when, in face, they are selling this product.

[DNF]

@kristoffer.carlsson

That’s probably a good idea in general.

I will keep that in mind.

[simonbyrne]

@nickeubank

But I’d never run into a situation where I couldn’t find an open source solution to a thing of general interest in Python land, in part (I suspect) because it’s built on a MUCH bigger pool of contributors.

Off the top of my head: Anaconda sells stuff, and PyTables had a Pro version for a while.

[Zach_Christensen]

@Tamas_Papp

when, in face, they are selling this product.

I think it was implied that it was hidden behind a pay wall. We’ve already discussed that much of the supporting work is already freely available but without the IT supported architecture. I remember reading the small page on JuliaComputing’s website about JuliaRun a long time ago and I came away with the impression that HPC in Julia required me to pay for JuliaRun (which was unfortunate as a graduate student with little sway in the purchase of that sort of thing). It’s a little clearer now but I think there’s still some room for improvement.

I think it might be nice if JuliaComputing had a table like they do for JuliaSure, where each row is a feature and each column is a level of availability. Then people could see that something like ClusterManager.jl is part of this and is generously supported by JuliaComputing for free.