What is it like being a (Julia) research software engineer?

Jobs
, , ,
1

Context

After defending my PhD in machine learning & optimization last December, I started a rather theoretical postdoc on similar topics. Now I’m wondering what to do next, whether inside or outside of the academic world.
I like reading scientific papers and learning or creating new things (algorithms, theorems, etc.), but I also very much enjoy programming. If my time spent on this forum is any indication, I am addicted to solving other people’s coding problems ^^ Oh, and teaching is an absolute blast, so I definitely want to continue doing that.

Question

Yesterday I say a job offer for a Julia research software engineer, and this led me to wonder: what’s it like in your shoes? How much creative freedom do you have? Is contributing to open source projects a big part of your work? Can you interact with or mentor students?
Some people I know think that if I go down the RSE road, I will have to transition to industry, with no hope of returning to a more classical research / teaching position. How true is that nowadays? Is it safer to bet on a research portfolio or dev skills, or can we maybe do both? And what’s the pay gap?

As you can see I’m a bit lost, so any and all advice is welcome!

16 Likes
2

Kind of a side topic but with this in mind, you may want to consider sales engineering. “Sales engineer” sounds like sales, but it’s really engineers on the front line helping customers understand capabilities and helping fix their real-world examples. For someone who likes working on someone’s real code and likes teaching smaller groups, sales engineering is something that should really be considered.

12 Likes
3

I think there is on the one hand a spectrum of what you want to do and on the other hand, jobs that provide this and the freedom.
Chris made already a good point with the sales engineer.

The “leaving academia means you do not come back” is still true in several countries I think, though (in my experience) stronger in Germany than for example here in Norway.

For interaction with students, you should stay in academia or quite close; for example in Germany there are the Fraunhofer institutes, which apply recent research in industry but are always affiliated / “close” to university and have students for theses – similarly here in Norway with SINTEF. I would consider these somewhere on the spectrum between classical research and industry.

For me personally (a German CS “Diplom”/Master, Math PhD, now AssocProf in Norway), this spectrum also holds to some extend. I use my packages in research for writing papers, lectures, supervising master students; but that leaves a little less. time for solving other peoples problems (besides the student’s ones).
So though I am the classical research position, I have the freedom incorporate software engineering into my research. I have to say though, that in Germany that was sometimes not so much values, here in Norway it ist.
So that might again also depend on the country you’re / aiming for and its academic atmosphere.

To the pay gap I can not say much, but I would usually guess, that towards industry the salary is higher, but the freedom is somewhat reduced.

2 Likes
4

I agree that the question is very dependant on the country you are in.

In France, there is two positions in universites that could come close to what you want

I’ll argue that the best shot you’ll get at everything you want is the “Maitre de Conférence” positions : there is a teaching duty and then a lifetime freedom to manage your own researches as you like, including software development. This is however a little hard to obtain.

There is moreover the “Ingénieur de Recherche” that is a little different: you have no teaching duties, and you are supposed to produce software and tools to help out the researchers from your lab. However, the content of the job is really varying from one lab to the other, highly dependent on the research teams around you: I knew one in a math lab that helped me on my first Julia project, teached me a lot of stuff; This guy has an independent research production as well. I know another one in an economics department that jumps from projects to project, never having a say in the research directions (and never getting authorship credit as well… although he implements most of the stuffs that produces the publication…). So i guess YMMV.

Last, there are research software engineers positions in the private sector, the content of which I cannot comment, as it highly varies: some persons in such private labs have time to do teaching at public universities, some dont. Some have a say in the content of the projects, some dont. Very few have enough power to choose their tools and languages (Julia).

4 Likes
5

I’m a software engineer in Scientific Computing Software at the Janelia Resarch Campus of the Howard Hughes Medical Institute. I end up using many progamming languages in my work: Java, Python, LabView, C, and of course Julia. I definitely appreciate the vibrancy of the Julia ecosystem, it’s packaging, and the ability to contribute to the core language itself. Many of these aspects are much harder in other language ecosystems when you want to build new packages.

Contributing to open source project is a large part of my job. Institutionally, we have a commitment to open science and open source.

Also, HHMI is self-funding so I’m not constrained by pressures from external grants. SciComp Software acts an internal software consultancy for the research labs. I have a fair amount of freedom to figure out how to complete tasks, but there is often a practical goal that needs to be accomplished. For example, I can use Julia for some tasks, but I may need the final implementation in Java. In research and scientific computing, you need to have some flexibility.

Research software engineers are in significant demand at the moment and the field is getting more organized. Compared to other academic positions, the pay can be competitive with junior faculty positions and is definitely better than most postdoctoral positions I have seen. Faculty positions involve a fair amount of fundraising, grant writing, and management responsibilities. If you are interested in setting the research agenda, then faculty positions are the way to go. If you are actually interested in doing the work yourself, then you may want to consider the RSE role.

We are not a degree granting institution, so we do not have many students, although there are some. There are some teaching opportunities for some of those in trainee positions such as visiting graduate students or postdocs. That said I’ve had the opportunity to mentor some technicians as they apply to graduate school as well as some of my junior colleagues.

The private research organization model provides something in between university-based academia and industry. Most of my colleagues think of themselves as academics, but they do not have any explicit teaching responsibility. Some do teach by partnering with universities or other groups. The point I want to make is that there exist a middle ground between academia and private industry.

Several of my colleagues have moved on from our Scientific Computing Associate program - basically a research software engineering training program - to a range of roles between software engineering and faculty positions in academia. Some of them are now in private industry. Others work for other research foundations.

I’m not sure what you mean by pay gap. You can definitely get paid more for similar work in industry but perhaps have less creative freedom. It’s really a question of what kind of salary do you want and what kind of position do you want in the future.

At Janelia, we do we have several open positions. Most of these job advertisements do we have payscales attached to them.

  1. Scientific Computing Software Engineer
  2. Open Data Specialist
  3. Scientific Computing Associate - Machine Learning
  4. Bioimage Data Analyst
13 Likes
6

Agreed with you @lrnv for France, I’ve the same feedback, once you leave research it’s extremely hard to come back.

I’m curious, if such a creative person like you @gdalle, eventually considers to make your own business ?

After all, academics is not really the best place to use your creativity (if I’m right), and teaching may leave you the feeling to miss something. With your Phd you’ve lots of opportunities in your domain to build a company at your image, and startup can be a really exciting place to work.

1 Like
7

My research group, the Quantum Sensors Group at NIST in the US, is looking to hire a talented research software engineer. We don’t have a job ad up yet, but if you’re interested please DM me.

Here are my answers to your questions based on how I think it will go in our group:

  • Creative freedom, we’re going to have some baseline requirements for maintaining and extending existing software, but beyond that there will be a lot of room for creative freedom and choices of new projects.
  • Contributing to open source projects would be a side effect, not a primary goal. To the extent that you can solve our problems by extending or writing new open source software, then the sky is the limit. But we’re not going to be excited about some unrelated work.
  • Interact with and mentor students, the answer is “yes, but”. We’re mostly a physics group, so most of our students and junior scientists aren’t going to be super interested in software topics for their own sake. That said, the software will be related to data acquisition and data analysis methods, and so to the extent that the the research software engineer owns those part of our projects, there would be a lot of opportunity to mentor junior scientists.
  • Teaching, we’re not a university research group, but some of our scientists have taught classes at nearby community colleges and universities. It’s definitely on the table, but not the default.
  • I don’t know about a “classical research/teaching position”, but I think you’ll have no problem finding long term satisfying work that meets a lot of your stated desires.
3 Likes
8

I would also encourage you to look at startups which work with Julia. An early stage startup will have plenty of problems in need of solutions, and the process of building those solutions can involve:

  • Creativity in finding a novel approach
  • Practical software engineering to refine that idea into the right implementation
  • Contribution to open source to fix what you need internally, which in turn drives more open source contributions/ideas
  • Building something new, which sounds in line with your desire for creative freedom

There are no students, but coming in with strong Julia expertise helps you to shape the codebase and technical vision. With strong Julia knowledge, there are lots of opportunity for teaching and helping within your team, although the interaction is necessarily different from the student/teacher relationship.

The PhD shows that you are capable of working in an underdeveloped space, and Julia expertise is invaluable in being able to realize your ideas. Startups are not academia, but the right position can offer lots of space for creative freedom alongside competitive compensation, in case that is important to you.

2 Likes
9

I’m an RSE at UCL so I think I’m somewhat qualified to comment on this (BTW, we are constantly hiring and Julia is explicitly as a desired skill in the job description).

Things can be very different depending on the arrangements of the group where you work, and the researchers you collaborate with. For example, my group is a central one, we work with the entire university, we’re allocated to projects depending on expertise, inclination, interests, availability, and these are reviewed every ~3 months.

Creativity freedom is usually large, but remember that you always need to refer to the people who bring the money, typically researchers with their grants, and sometimes they may have strong opinions on the tools (languages, packages, etc…) to use, but not necessarily. I use Julia very little, although I managed to successfully suggest using Julia for one or two projects where the researchers were considering other languages. Also, note that at least in the UK, but probably elsewhere too, RSEs can apply for research grants, so one can effectively pursue their own research projects and choose the entire stack to use, if funded.

Keep in mind that other university may have different arrangements, RSEs may have a fixed-term position to work on a single project for which the software stack has been already decided, which may be the case for the post you saw.

Again, this ultimately depends on the PI of the project, but in general in our group we try to suggest using and developing open source packages whenever possible.

We can mentor student, from bachelor to PhD level.

I know a few people who have been RSEs for a 1-2 years and then went back to the “proper” academic career, that’s definitely not impossible. In our department there’s a career track for progressing from grade 6 (PhD level) to grade 9 (associate professor level), in principle also grade 10 (full professor), but that’s harder to achieve right now. If one’s aspiration is doing more programming than hands-on research while remaining in an academic environment, there’s probably little interest in moving to an academic career.

At least in the UK, RSEs are on academic salaries. If you see the difference between academic and industry salaries, you get the pay gap :stuck_out_tongue:

9 Likes
10

I self-describe as an RSE at least for part of my job.
I work in an industry R&D department, creating models and simulations for combustion, flow, and systems modelling.
I help maintain, improve, extend two old codes, C++ and Fortran, respectively.
I also help a bunch of scientists and engineer who write some code as part of their duties make sure that the result is maintainable, testable, documented, and try to spread SWE best practices, tooling, and informally teach some Python, etc.

From time to time we have students doing their thesis here, but there’s no formal involvement with an uni or somesuch.
I could teach on the side if I wanted, but I don’t find the time anymore.

I typically have lots of freedom in my technical choices, and can introduce new things (although the I also have to evangelize a bit :grinning:). There’s no Julia here (yet?), so far it’s not been a good fit to the things we need to create (deployable/small binaries, quick-running GUI tools), and nobody else knows it. Maybe that will change at some point, I try to follow develoments. :man_shrugging:

2 Likes
11

Consider a permanent position as IR (ingénieur de recherche) in France, even on topics outside IT but where they need an IT guy for modelling/simulation/computations on domain X.

IR have some teaching, but are really free to develop their own agenda without all the teaching burden of “maître de conference” and all the “publish or perish” of “proper” researchers…
Also, it depends on the laps, but in mine they are very much at pair of researchers, perhaps more applied on a computational platform… I am writing from a train coming from a 6.5M€ project meeting where the project coordinator in an IR, or another IR that is the director of its lab (but I also know labs where the IRs are considered “support staff”… so it is a sort of a not well defined joker position that means different things in different places…)
If you then get the HDR (Habilitation à diriger des recherches) you can mentor PhD students.

With all your experience you would strike any competitor… :wink:

(sheep, we got a position as IR that closed few weeks ago :-/ )

1 Like
12

FWIW I made this jump about a year ago. PhD in math+ML; now working in big tech wearing multiple hats covering both software and modelling.

To work through your question in order:

  • Internally I’m tackling problems I think are important, and my contribution to this is primarily writing and open-sourcing numerical algorithms. (In JAX in my case, not Julia.) This is great.
  • Creative freedom: a lot of freedom. I found a tech leadership role. As such I’m able to define the problems I work on, the tech stack we use, etc
  • Contributing to open source projects: I really wanted to open-source a lot of my work (Equinox; Diffrax, jaxtyping etc.), and this is supported. Like you say, it’s fun being part of a community and “solving other people’s coding problems”.
  • Mentoring students: I’m also supervising a PhD student, so yes.
  • Transition back to research: part of my job is still doing research. I definitely find less time to write papers than before, though. I imagine I could probably transition back to full-time research if I wanted to. (In industry research groups certainly; academia maybe.)
  • Pay gap: unsurprisingly big tech pays much better than academia.

So it is totally possible to find a job doing RSE-like-things with the freedom you’re after.

I think the take-away is to make your needs clear when you go job hunting. I took this role because it offered me the above freedoms, and I was up-front that they were going to be a condition of me joining anywhere.

14 Likes
13

Great advice for all job seekers!

1 Like
14

I am not an RSE per se but close enough, a product engineer at PumasAI. I use Julia pretty much on a daily basis. At PumasAI, we develop and sell software (written mostly in Julia) and we do consulting for pharmaceutical companies. Some of the products and consulting projects we do require a heavy dose of math and CS (statistics, dynamical systems, machine learning, optimisation, automatic differentiation, etc.). Research and publishing is encouraged especially in the flagship conferences and journals in the field where publishing also counts as marketing. But publishing is not a priority in industry so I often struggle to find the time to write papers but it’s not impossible to do at least 1 or 2 papers a year if you are committed. We have lots of room for innovation and creativity if your creative ideas can solve some real problems people have. Usually the product needs are dictated by our clients’ pain points and the needs of the consulting projects. Finding new ways to solve these needs is encouraged, but if there are satisfactory methods out there already then implementing them in user-friendly software is also a fine solution. Personally, I like the option of being able to do regular software development (albeit somewhat math-heavy) for a living, while also being able to get creative and do research-like activities when I want to or if the project/product requires it.

As for academia, I am currently a research affiliate at the University of Sydney where I collaborate with academics there. I also have friends who are academics and we often do slow-paced research and publish papers together (e.g [2201.11926] Simplifying deflation for non-convex optimization with applications in Bayesian inference and topology optimization and [2305.05080] Earth Movers in The Big Data Era: A Review of Optimal Transport in Machine Learning). However, any research that’s not related to PumasAI, I do it in my personal time so the pace is often very slow because it has to take away from family and personal time. So it takes commitment to do that. I enjoy doing research though so I just look at it as a hobby :slight_smile:

4 Likes