It’s great that you guys investigated this and made a suggestion to TIOBE, but because of its methodology, it remains one of the noisiest measures of programming language usage/popularity.
Composite measures like Redmond and IEEE Spectrum are still a bit shaky, but at least they are better equipped to catch measurement anomalies (eg one component experiencing wild fluctuations that does not show up in other components).
In the long run, a better solution could be a composite index with
a transparent data collection process (all scripts open source),
raw data from this archived with open access,
with a latent process estimated using a robust statistical filter to produce reports at some frequency (this step is crucial, otherwise the noise swamps all trends up to medium run).
I know this is not for the Julia community to solve, but perhaps NumFocus could get behind something like this.
I would never used these search terms unless I don’t know anything about Julia and want to find out what it is. It does not make sense to measure that unless it consider the complete history.
Once I know julia, the most common search terms are “julia xxx” where xxx is anything that helps me find a package, or “julia yyy” where yyy represent so a programming technique such as thread, performance, etc.
I think that particular ranking (TIOBE) is best taken as a measure of external curiosity about a language–how many outsiders are interested in the language rather than how many users the language has. The two are correlated but distinct, and both are useful things to know.
Well, it isn’t counted how frequently these terms are searched but how many hits there are. I think most websites writing about Julia should have “programming” or “language” in there somewhere as well, so it makes sense (to me at least).
Note that the quotes in "Julia programming" are actually part of the search term. It’s not just Julia somewhere and programming somewhere — it’s the exact phrase Julia programming. That’s why adding "Julia language" makes such a big difference… but I still never search for either. It’s either julialang or jl that most successfully bring things up for me.
You are right, that limits it somehow. But unless I’m mistaken in how the algorithm works, it doesn’t matter if you or anyone searches for that term, only if there are hits, aka websites with these phrases to be found. And I think the latter is much more likely than the former.
Note that TIOBE does not look at how often people search for “Julia programming”, it measures how many web pages search engines report to have that exact phrase in them. So this is not related to how much people search for a language, but how much content is out there on the web with that phrase, even if no one ever looks at it. There’s pretty obvious issues with that as a measure (if there a billions of pages but no one ever reads them, so what?), even assuming that search engines reported accurate numbers, but there’s a lot of evidence that search engines essentially just return random, highly variable numbers for “hits”. If TIOBE wants to remain relevant as a PL popularity metric, they should probably get ahead of things and include different metrics, but that feels a bit unlikely to happen.
Yeah, sorry, didn’t mean to send things down the wrong path here — I mentioned which search queries work for me as a proxy for what appears on relevant webpages, but of course it’s a really bad way of measuring that.
From the answer I got from Jeff years ago (at his thesis defense), it was built to handle the unique needs of scientific/technical/mathematical computing, but it was NOT built just for that community.
It was meant also to be a great general purpose language (which it is - IMO, if it were not so, it wouldn’t really solve the “two language” problem that had so frustrated the original creators)
The fact that most of us use “julialang” or “julia language” rather than “julia programming” in google search is already a huge bias in the ranking index.
I also want to point out that some ranking uses the number of question/answers on stackoverflow. And the very existence of this discourse is actually impacting this activity. I know some of the newcomers are cross posting in SO and discourse. But generally, discourse is the prefered way to ask a technical Julia question.
As someone who is active at both places — and as a moderator/admin here, no less — I strongly disagree. The two sites have very different purposes.
Now, yes, at Discourse we do have a significantly larger community of advanced Julia programmers, and yes, the universe of questions that this site is equipped to handle is significantly larger than those that make good StackOverflow questions, but SO is exceptional at answering tightly focused questions and providing highly relevant fodder for google searches in the future. It can sometimes be tough to grok what exactly the answer is in a old thorough and technical discourse thread that you stumble across, but on SO things are structured to be much more straightforward (on both sides — both questions and answers).
In fact, a number of us are slowly working on mining discourse for good candidates for SO questions — that’s where some of the recent self-answered questions are coming from. I encourage those of you who have well-received “answer” posts in discourse threads to consider moving them to SO, especially if you’re finding they continue to receive s long after you’ve posted them. This isn’t just to superficially improve our rankings on arbitrary metrics, but it’s actually helping to satisfy a real need we have: lots of programmers rely upon SO answers popping up in google searches to learn new languages, and Julia’s presence there is indeed a little anemic.
I didn’t mean that SO and discourse fulfill the same purpose. I fully agree with what you said.
Basically when you want to reach a community, especially when the community is rather small, finding someone skilled enough to understand your problem and give you a piece of advice is hard. So multiplying the channels multiply the chance to find that person. But sometimes it doesn’t make sense to cross-post. As you said, some technical questions bring fruitful discussions on discourses cannot be published directly to SO, because that’s not why SO is made for.
But some questions can be reposted and should. SO is still the most prefered way for programmers to find a solution. Clearly, don’t post on SO for ranking, do it to help people!
We’ve had a programmer start using Julia full-time three months ago, and that is his #1 complaint: that Google/stack overflow doesn’t give him the answers to everything. Discourse just doesn’t appeal to him, and I can understand…
FWIW, “Doesn’t appeal to him” doesn’t necessarily imply “won’t use it”.
Clearly, the help one gets here on discourse is awesome. But it’s a fact that people like to use the tools they already are familiar with. If you haven’t seen a discourse forum before it might not appeal to you, simply because it’s different from what you know, like SO. Not a big deal on the long run, as you will get used to it, but initially it might hinder adoption a bit.
I think what most people like about StackOverflow is that they can find the solution to many of their problems without asking the question themselves. Most of the people I know don’t even have an account, yet they use StackOverflow all the time.
For julia there aren’t as many questions already asked as other languages, for several reasons (relatively new and little used, other help channels available, etc).
s long after you’ve posted them. This isn’t just to superficially improve our rankings on arbitrary metrics, but it’s actually helping to satisfy a real need we have: lots of programmers rely upon SO answers popping up in google searches to learn new languages, and Julia’s presence there is indeed a little anemic.