Tamas, I think there is a fundamental difference between your view as a developer/programmer and that of a non-programmer user like myself. Users often merely want to be able to install a package, call a function and extract the results. In most cases this can be done inefficiently through the REPL. Over time, users can develop more efficient approaches to automate discrete processes, whence the move into programming. Beginner users like me want to know what to do about a TypeError without needing to know the underlying architecture that makes it so. In my hack approach, strategies such as redefining input variables as the correct type, converting types or revisiting the function documentation have proven viable, but this understanding was hard-won after many hours reading through the manual, working through examples and trying different commands.
You must have misunderstood something — I am not a programmer. I am an economist who picked up some programming. I am no more of a programmer than a mathematician (because I know some extremely basic math) or a typesetter/graphic designer (because I know LaTeX). Most of the people here are scientists in some field who are in the same situation — it is a skill we picked up.
Yes, I am fully aware of this. Very frequently, people just want to do one thing and not invest too much in the background.
I just don’t think this approach is viable. Inevitably, you will run into the next thing you just want to do, and the next one, over and over. It’s like saying that one does not want to learn play the guitar, just play this particular song. I don’t think that’s how it works.
Yes, that’s how learning something complex works. Think of a subject you found difficult (but useful) at university. Learning Julia will require the same amount of investment.
Sorry if this sounds like lecturing. I fully sympathize with the situation of people who just wanted to do something they considered basic and ended up spending a lot of time going down various rabbit holes of the type system, multiple dispatch, inference pitfalls, and packages.
I just think that the whole process is inevitably part of learning something complex yet powerful, and approaching it with this mindset will make it a bit easier and a lot less frustrating. It is like trekking long distances: you know it will be difficult, but also that you will get there eventually.
(I was recently reorganizing my bookself and found my copy of Rudin’s Principles of Mathematical Analysis. I learned introductory analysis from this book in the first year of university. From the margin notes I made, it must have been an immensely frustrating experience. Yet somehow I remember it as a rewarding one. I think that’s how it works out for most people with Julia.)
Thank you, these examples are helpful. It’s quite interesting to me what large variety of things people can be confused by 
I remember helping friends with an R programming assignment. I didn’t understand why they had problems, until I noticed they hadn’t understood yet that assignments work from right to left… That might seem silly but it just shows that for someone with more knowledge it can often be hard to understand where a beginner is going to struggle.
I’m not advocating against good documentation, this can be key to a deeper understanding. But I also know this:
- even if you read the whole documentation beforehand, you can’t remember many critical details
- these details trip you up while you’re coding, when you’re not in a mindset to read documentation
- the maintainer of the package probably knows these critical details very well and has documented them nicely
- details in the documentation that you don’t find easily when you need them are not useful
I’m arguing for any kind of way to make these details come to the user while they’re coding, especially with critical details that are non-obvious for casual users. Put these things into error messages already or write guardrail methods that catch incorrect inputs and explain why they’re incorrect.
Is it better to get a MethodError somewhere deep in a stack trace, googling and finding after half an hour that there’s a paragraph in the documentation describing why you should only put Float values into a function and not Ints, or is it better to put that paragraph into the error message of a method that catches Ints and tells you immediately why that is wrong? Especially if you can already foresee that a user might make that mistake frequently? I’ll always advocate for the second way. As package maintainers we should put all this implicit knowledge that we have into our code so that it “speaks to the user”, not into the documentation (of course you can put it there as well). That’s what I mean by hand-holding.
(This is especially important given the generic code people write in Julia. Your functions are going to be used with all kinds of inputs, so really make the underlying assumptions clear, and your error messages informative)
This all depends, and I am not sure what you are advocating is generally better. Consider
function f(x, y)
z = g(x, y)
h(x, y, z)
end
g and h here are functions that have a reasonable default, but are exposed, documented, and the user can define methods for them (this is quite common in Julia — eg if x and y can be Float64, a lot of code can work with ForwardDiff.Dual without problems).
Should f validate or restrict its inputs x and y to what is envisioned as a typical application (eg Float64)? If the goal is to keep the code generic, no. Doing that would preclude extending f via g and h indirectly, allow for the docs and the code to become out of sync, and hamstring the implementation for no good reason.
In my experience, a lot of code is just inherently not generic over all parameters. So restrictions and helpful error messages in those cases would be mostly useful I’d say. Also, if you keep the catch methods generic enough you can still overwrite them as a third party with more specific methods that allow your desired functionality again.
Hey @jules, I’m interested how you’re going to approach your tutorial for beginners. As a grad student in a neuroscience lab myself, I see many people basically “porting” to languages such as Julia. In our MATLAB dominated institute, most people when introduced to Julia will probably just try to rewrite code written in MATLAB. This may be okay at the start, but eventually it just translates to “doing MATLAB in Julia” where people don’t really use the style or strengths that the language is designed for. After that, difficulties ensue (MATLAB works right out of the box nicely) and everyone goes back to square one. I personally haven’t developed how to code in Julia effectively myself, though I am slowly going through tutorials and documentation. Still, I am at a loss when people ask how to go about learning Julia—with @Tamas_Papp’s suggestion being too rigorous for most people. Not because it’s ineffective, but rather because it seems too steep for most people.
Good question, I haven’t made up my mind, yet. I want to showcase the possibilities you gain in Julia vs Python or Matlab but have to accept that many people don’t want so many possibilities. They want to know if Julia can make their life easier right now, and without a lot of effort on their side. And maybe the answer to that is just no, as much as I would like it to be yes. As this is often not about the base language but about packages that the whole field uses (SPM in Matlab for fMRI analyses) the discussion might stop at “do you have SPM or a better alternative in Julia”. (No but you could write your own in Julia that’s soo much better… Yeah right )
If someone really depends on a very specific but rather complex tool that is hard to replicate, then I think it is just best to acknowledge that at this stage, they may not benefit from switching to Julia. I think this is OK: new languages are of course at a disadvantage when it comes to mature libraries. This will change only gradually.
Contrary to what you suggest, I don’t think that casually suggesting that one reimplements these complex tools in Julia is a common response in these situations. People in the Julia community are usually aware of the cost this implies, which is why various interface packages to other languages were created in the first place.
I meant this jokingly as something I would like to reply, but know that it doesn’t make sense. A lot of the established tools would have benefitted from Julia if it had been around at their inception
It is certainly possible to make error messages more beginner friendly. The Elm Language is one example of a language that goes to great lengths to make error messages readable and beginner friendly. Here are a couple examples of error messages that the Elm compiler provides:
> type Fruit = Apple | Orange | Kiwi
> fruitToInt x =
| case x of
| Apple -> 100
| Orange -> 200
|
-- MISSING PATTERNS ------------------------------------------------------- REPL
This `case` does not have branches for all possibilities:
4|> case x of
5|> Apple -> 100
6|> Orange -> 200
Missing possibilities include:
Kiwi
I would have to crash if I saw one of those. Add branches for them!
Hint: If you want to write the code for each branch later, use `Debug.todo` as a
placeholder. Read <https://elm-lang.org/0.19.1/missing-patterns> for more
guidance on this workflow.
> fruitToInt x =
| case x of
| Apple -> 100
| Orange -> 200
| Kiwi -> "300"
|
-- TYPE MISMATCH ---------------------------------------------------------- REPL
The 3rd branch of this `case` does not match all the previous branches:
4| case x of
5| Apple -> 100
6| Orange -> 200
7| Kiwi -> "300"
^^^^^
The 3rd branch is a string of type:
String
But all the previous branches result in:
number
Hint: All branches in a `case` must produce the same type of values. This way,
no matter which branch we take, the result is always a consistent shape. Read
<https://elm-lang.org/0.19.1/custom-types> to learn how to “mix” types.
Hint: Try using String.toInt to convert it to an integer?
Wow that’s elaborate, I like it
This kind of error message is definitely too much.
It is alright to better explain the error, but it is also important to keep it brief so that non-beginners can pinpoint what slipped their infallible programming skills, just by reading a one-liner instead of a whole chapter about basic syntactic definitions.
I realise that maybe the absolute beginner is not one to go search in the documentations, and i understand it. But then i also might argue that maybe a young, not yet adopted by many, language such as julia is not really for that kind of audience.
Picking up a well established language such as python, R or MATLAB, for an absolute beginner is possible without ever reading a single line of documentation. Basic tutorials to get you started and StackOverflow questions to fill in the gaps and those who respond to such questions are usually those that actually read the documentations and can pin point the exact sections for you, which is wonderful. But for languages such as julia, those kind of easy to get, already digested informations are not so easily available.
If you want to be ahead of the curve, you have to put in some work, sadly.
With this said, i definitely agree that some improvements in the error logging could be made, but more toward better clarity than better hand-holding (maybe for that some kind of special flag like --verbose-beginner).
By using Julia to solve a problem(s) at hand - usually following some blog post series. Hopefully, the blog series is showing how Julia allows you to do ABC intuitively, naturally or the way you’d expect, but that Z, Y, or Z make obscure, tricky, or prohibitively slow.
There are many such domain specific blog series that substitute for RT*M in various languages (Ruby and web site creation come immediately to mind in terms of successful efforts).
Here is an example of a Julia series: Constraint Solver: Dealing with real coefficients
Writing such series is another matter - might be suitable for a GSoC project? - A series that cover the 200 pages referred to.
But, in my experience, these seem to rely on domain specialists who are Julia evangelists (such as @jules ) to trawl through the 1247, 400, 200 pages and distill what, in their mind, will be salient to their domain audience.
The use of domain specific problems that are relevant to the audience provides and sustains motivation to push on. Formulating those problems in a way that makes language shine isn’t always easy - and should also be frank about exposing any shortcomings/sharp-edges so newbies aren’t bitten
Note that Julia Computing offers free training material and 3x4hr (paid) webinars for that accomplish some of this (if you’re into image processing or AI/ML these are well suited to you).
I’ve taken the 3 sets of paid-webinars - they seem to play this role, and you pick up explanations, tips and gotchas that you’d expect - the ability to ask questions of an expert is the main benefit.
I expect the free training material will also alert you to pitfalls and some topics where “hand holding” is required.
While domain-specific introductions can be very useful, I consider them complements, not substitutes of the manual (or equivalent, eg a general intro book on Julia, though I still think the manual is best).
I don’t think such a distillation is feasible: there are no domain-specific subset of the language proper. And domain-specific introductions don’t actually aim to do this: they usually demonstrate a subset of Julia, but I have not seen any of them claim that this is all that is required for a particular domain.
(Incidentally, I clarified above that I am not talking about 1200+ pages, so I am surprised you considered it necessary to interject it into this discussion again).
I realize the above is from a week ago, but I think it’s still relevant. For me, it would probably take more than a day or two, because I like to proceed slowly and with lots of side-experiments in the language. However, I do not see the problem with that and - as a beginner in this language - agree mostly with what you have written in this thread so far.
A basic skill like calculus is something that most here have probably picked up at some point during their lives, and it likely took more than two days to do that. This is fine, because many use that skill (explicitly or implicitly) all the time. Similarly, if programming is a substantial part of one’s daily work, then I would think that 2 days is actually a very small cost.
Regarding domain-specific introductions: I think they are great, because they can provide motivation, but they tend to fall short because they hide either details (not such a problem) or general structure (a problem). So far, I found the manual itself indeed quite nice, also because of the clear separation between “Manual” (what I am reading) and “Base” and “Standard Library” (what I am referring to only when needed).
If you’re presenting Julia to R users, I think you may want to spend a little time explaining virtual environments and how the package manager handles them.
In R, we are used to
install.packages("foobar")
Then, from any script anywhere on our machine, we can call
library(foobar)
and have access to the foobar library. Because Julia uses virtual environments, that’s not how things work. As an R user, that took me awhile to figure out. Of course, Python users will be less confused by this.
As an R user, I think this was the only thing that really tripped me up coming to Julia, which is speaks quite well for Julia. My guess is that I have a good deal more programming experience than the typical R user though (especially grad students).
That is how things work if you only ever use the default environment.
I think the environments can be a little confusing at first if you come from the “everything I install is just available everywhere” mindset. But then when you ask them, people often already had problems because of mismatching / conflicting versions of packages, or they had trouble getting their own old or someone else’s code to run. They just don’t know that there’s a solution for that and it’s environments. I’ve had the experience that even non-programmers were quite interested to know more about this topic once I explained the gist. Scientists always respond well to better reproducibility 
@jules: yep, my comment was definitely not a “here’s a problem with Julia”, but a “here’s something that R users may struggle with”. Very different categories
No worries, I got that, I just wanted to add some perspective I think you’re right that it’s important to stress this part, so I’ll definitely include environments in my tutorial.