Planning to launch an annual Julia developers survey

Folks,

We (at Julia Computing) are planning to put out a survey of Julia developers. We hope to put it out annually roughly at this time of the year, so that the results can be presented at JuliaCon.

Here’s the survey that my colleague Andrew Claster has prepared, and it would be great to receive feedback on it. Please provide your feedback in this thread, or as comments in the document.

https://docs.google.com/document/d/1HjLSY5DBnZ4PZWJVA5fLaOLXDCGLu86SAXTJ5qdKXYI/edit?usp=sharing

The formatting is unimportant since it will be converted into a different form and sent out.

Thanks,

-viral

8 Likes

I think it would be interesting to have a column for added ot the language grid to know if survey respondants previously used the other language more than they do now.

E.g. I want to know how many julia programmers are Ex-Matlab programmers,
vs Ex-lisp programmers

4 Likes

I’d probably change this:

IDE support (Juno, VS Code)

To:

Editor support (Juno, VS Code, etc.)

3 Likes

I think there should be a question for what is the first version of julia you used.
Which should be annotated with version and year.
So we can find out how long people have been around.

Rather than:

  1. When did you first start using Julia? ENTER YEAR BETWEEN 2011 AND 2019

It is easier for some of use to remember e.g 0.3
and that conveys the same information?

But I could be wrong here. Maybe just asking year is fine.
Can always look it up

12 Likes

The following questions are for statistical purposes only.

What does this mean?

1 Like
  1. Which editors or IDEs do you use most frequently? MULTIPLE RESPONSE - PLEASE SELECT ALL THAT APPLY

This should just say “frequently”, or should be Single response

1 Like

LGTM. One small remark, question 11, option 7 would look better with vi/vim (vi is somewhat obsolescent…)

You might want to add a line about compilation times and time to first plot issues, otherwise people will certainly add it in the other/specify class, which will make it a bit harder to collect together.

2 Likes

Also might be useful to add a “what area in julia or its ecosystem do you think needs work”, with broad areas (eg tooling, autodiff, threading, distributed, etc)

5 Likes

Excellent idea. I really liked the questions! I also agree that it would be fine to have a question about what the programmer thinks that must be improved in/added to Julia.

I would go further and change that to Vi/Vim/Neovim :smile:

  1. How did you download/install the Julia version you most frequently use? MULTIPLE RESPONSE - PLEASE SELECT ALL THAT APPLY

This should probably read “the Julia versions you most frequently use” or be single response.

There might also be a companion question like “How would you like to be able to install your next Julia version?”

  1. Do you identify as underrepresented in science or computing because of your: MULTIPLE RESPONSE – SELECT ALL THAT APPLY
     
    11. None of the above

Should “None of the above” be interpreted as:

  • “I don’t identify as underrepresented”, or
  • “I identify as underrepresented in science, for reasons that are not listed above”?

This might be clearer if both options were clearly stated.

3 Likes

Should there also be a question regarding fluency in (natural) languages? This might perhaps be useful e.g. to correlate with documentation issues.

2 Likes

Who will have access to the results? Is this specifically being run by Julia Computing vs. some community team? Will the results be used for the benefit of Julia Computing, to the exclusion of competitors or other for-profit entities who would benefit?

4 Likes

I hardly get to use Julia these days because we’re pretty locked into R at work and I don’t really program outside. An elaborating question for those who answer <100% to question 8 might be useful. Something like “For which of the following reasons do you not use Julia more often?”

  • I use other languages because of pressure from clients, collaborators or colleagues
  • I use other languages because of company policy
  • I use other languages because I require more stability
  • I enjoy using or exploring other languages
  • I do not feel like I know Julia well enough to complete some programming tasks
  • I do not feel like Julia is well suited for some programming tasks (specify)
  • I require packages, libraries or tooling that is only available or is more complete in another language (specify)
  • [etc]
  • Other (specify)

Kind of similar to question 4, I guess, but I think the framing will get people to name specific reasons they haven’t used Julia elsewhere rather than just identifying things that might not be perfect.

3 Likes

As @oxinabox suggested, question 1 makes it difficult to convey that you previously used 5 languages for different things and now use Julia for everything - it will just look like we never used anything else. One question could be “How frequently did you use these other language”

1 Like

Question 3: is there a clear difference between speed and performance?

Suggestion for question 6: “because it solves the 2-languages problem”

Question 28: the difference between race and ethnicity may not be understood the same by everyone, and mixing them in the same question seems weird to me, for example if I choose both (1) (White, Caucasian, European) and (4) (Black, African, African-American, Caribbean), you really have no idea what is my skin color nor my ethnicity.
Given that the concept of “race” is quite controversial, I would remove it from the question.

1 Like

I think it would be much easier to use some kinds of online survey websites to get this done, which makes it easier to aggregate survey results.

1 Like

I think this may reflect a cultural difference between the US and other parts of the world, especially Europe. In the US, race and ethnicity is a standard part of various government-sponsored surveys, eg the CPS, and is usually understood by the respondents, while in (some parts of) Europe it is a more sensitive issue for historical reasons.

In any case, it is only meaningful to ask about these things above a certain expected sample size (say, 10k–100k), which I guess may be tricky. But I understand the motivation to collect demographic data anonymously, as it is informative about the inclusiveness of the community.

1 Like

I have removed the ability to comment on the survey, since some people were filling out the survey in here and making formatting changes - making it hard for me to separate these from substantive comments. Please chime in here.

I’ll be freezing it in the next couple of hours.

Thank you,

-viral

1 Like