Blog post about my experiences with Julia

[Volker_Weissmann]

@Henrique_Becker

@Volker_Weissmann

@includeonce

From where this comes from? Is it Base Julia?

No, it’s from GitHub - tgflynn/IncludeGuards: Provides an include guard mechanism for julia .

@Henrique_Becker

@Volker_Weissmann

Another thing is that dynamic typing (without lots of type assertions) is kind of a violation of this guideline.

This is kinda of a good point but moot. The way Julia is designed asks for you to be as lax as possible with type restrictions in function signatures, what does go against the “fail as soon as possible” school of thought. It is not a problem without workarounds, but these workarounds must be explicitly written instead of just using the type system to do the job. All in all, it is the trade-off that any dynamic language makes at some level.

What would you suggest to debug this code/make the error message nicer?

MethodError: no method matching copyto!

[Volker_Weissmann]

@goerch

Nice, so you are that kind of guy (as I am or was?). Try some dependent type programming and you might come back to Julia someday to have some fun, as I did.

Never used dependent types or read about them.

But yes, take a cliche vegan, substitute vegan food for rust, that’s me. Every second argument is: “rust does this better”.

[goerch]

@Volker_Weissmann

Never used dependent types or read about them.

Not sure if I’m qualified to talk about this, but take a look at Idris.

[Volker_Weissmann]

@jakobnissen

• We don’t have good autogenerated docs, like cargo doc . It should be possible to create a program that reads Julia code, lists exported functions and types, which functions operate on types, etc, and links them together and create a HTML file. That would be a really nice addition to Julia.

You have Home · Documenter.jl . Go to JuliaHub and click on the documention links.

@jakobnissen

Things that are not so bad

• Having typeof(x) give the whole type with tonnes of type parameters is not a problem, IMO. You can always just ask for the base type. See also https://github.com/JuliaLang/julia/issues/35543
• apropos("Dict") is not the way to ask for methods with Dict . The right way is… methodwith .

But methodswith(Dict) also returns addenv.

@jakobnissen

• You can redefine functions using Revise

Run

using Revise
function func(x::Int)
    print("old")
end
# function func()
#     print("new")
# end
func(123)

then switch the comments:

using Revise
# function func(x::Int)
#     print("old")
# end
function func()
    print("new")
end
func(123)

and it still prints “old”. There is lingering state, which is bad.

[Henrique_Becker]

@Volker_Weissmann

What would you suggest to debug this code/make the error message nicer?

I do not know SimpleDiffEq enough to make a substantiated suggestion. If I understand the problem correctly you did pass u0 as a Float64 but then in a hook you did use du .= 2.0 * u (where du starts with u0 value). So it is a little strange to me how did you arrive at this situation (you did not have any mutable container defined in the code, why .= was used then?).

All in all, in fact, I do not find the error message to be bad. It does point to a line of your code, and it is precise about the problem, the issue is that with the broadcasting machinery involved you get names like materialize! and copyto! that mean nothing to the novice programmer. It does not seem to me a problem related to what we were discussing (failing as soon as possible) but the fact that broadcasting error messages are confusing to novices. It would be impossible to fail sooner there because the code takes a generic hook (if I am not wrong), and as so it cannot check the input type against anything. This is a common problem with hooks/callbacks, not a language problem, even if some languages like Haskell were able to do some very aggressive trade-offs that may solve this problem of designing code with hooks/callbacks.

[lmiq]

@Volker_Weissmann

and it still prints “old”. There is lingering state, which is bad.

Uhm… no, that is because you are not tracking the changes there. If you put the function in a file (or module), Revise tracks the changes:

julia> using Revise

julia> includet("./teste.jl")

julia> f(1)
1

julia> f(1)
2

where teste.jl is:

#f(x::Int) = 1
f(x) = 2

[suavesito]

@Volker_Weissmann

There is lingering state, which is bad.

I mean, that state is just two different methods of the same function func. Revise will not “delete” methods of a function, it will just allow to “reload” methods definitions when you’re editing a file that is included.

[Volker_Weissmann]

Thanks, cool, I always just did shift+enter in vscode.

[Volker_Weissmann]

Yes. But this can make for a really funny debugging session.

[jakobnissen]

You have Home · Documenter.jl . Go to JuliaHub and click on the documention links.

Yes, but AFAIK, Documenter’s autogenerated documentation is not very thorough. It adds the docstring. It doesn’t for example, add links to functions automatically found with methodswith, or tell which position in the type hierarchy a type is, or which type parameters a type has.

Julia has pretty good human-written docs, but I think Rust shows the autogenerated docs could be better.

and it still prints “old”. There is lingering state, which is bad.

Revise doesn’t track the REPL. Maybe it should? But if you do what you did there in a file that is tracked by Revise, then it will actually print new, not old.

[lmiq]

@Volker_Weissmann

Thanks, cool, I always just did shift+enter in vscode.

Indeed, that was something I mentioned right above in some place: I think that finding the right workflow in Julia requires some guidance. I taught courses in Julia, and it was completely different when, on the second time, I started by showing the students what I think is the best workflow, before anything else. Without that people can get a very bad first impressions, and the good workflows are not obvious. It is a barrier that must be overcome.

[ChrisRackauckas]

@Volker_Weissmann

What would you suggest to debug this code/make the error message nicer?

MethodError: no method matching copyto! · Issue #48 · SciML/SimpleDiffEq.jl · GitHub

@Henrique_Becker

All in all, in fact, I do not find the error message to be bad. It does point to a line of your code, and it is precise about the problem, the issue is that with the broadcasting machinery involved you get names like materialize! and copyto! that mean nothing to the novice programmer. It does not seem to me a problem related to what we were discussing (failing as soon as possible) but the fact that broadcasting error messages are confusing to novices. It would be impossible to fail sooner there because the code takes a generic hook (if I am not wrong), and as so it cannot check the input type against anything. This is a common problem with hooks/callbacks, not a language problem, even if some languages like Haskell were able to do some very aggressive trade-offs that may solve this problem of designing code with hooks/callbacks.

FWIW, I think this is pointing at something I’ve been meaning to open an issue for. Broadcast should really catch errors and throw a higher level error. It really can. Saying “I cannot materialize!” assumes people know some deep stuff. materialize! is not even one of the documented interface functions for Broadcast:

https://docs.julialang.org/en/v1/manual/interfaces/#man-interfaces-broadcasting

So yes, it should catch this and say “In-place Broadcast was not possible with types X, Y, Z.”. And then, even better, if it’s trying to materialize an in-place broadcast to a Number, it should catch that and have an explanation as to why x .= y is never going to work with x = 2.0.

That error message can definitely be better.

[Volker_Weissmann]

@Henrique_Becker

This is a common problem with hooks/callbacks, not a language problem, even if some languages like Haskell were able to do some very aggressive trade-offs that may solve this problem of designing code with hooks/callbacks.

If you are using the greatest Runge-Kutta implementation ever written you get really nice error messages.

[Volker_Weissmann]

@jakobnissen

Yes, but AFAIK, Documenter’s autogenerated documentation is not very thorough. It adds the docstring. It doesn’t for example, add links to functions automatically found with methodswith , or tell which position in the type hierarchy a type is, or which type parameters a type has.

Julia has pretty good human-written docs, but I think Rust shows the autogenerated docs could be better.

Absolutely

[Volker_Weissmann]

@lmiq

I started by showing the students what I think is the best workflow, before anything else.

Maybe what you showed them should be here.

[suavesito]

Well, interactive code editing/debugging in Julia has always been nice from my point of view.

Debugger.jl and some interactive utils like @less are really cool for debugging and exploring code. I had have no problem with them missing the right place to things I need.

You can see this video from @miguelraz about nice utilities available in the REPL about interactive utils.

[lmiq]

@Volker_Weissmann

Maybe what you showed them should be here.

Yes, it should, but in Julia it really needs to be pretty much stressed the first time. The good workflow avoids a lot of the Julia development pains (most TTFP, high level use and debugging, etc).

In some sense it its a “problem” because the tools needed (VScode probably in most cases - other editors can be used, but to guide people better stick with one - , and Revise) are not tools of Julia itself. But I think that the language would benefit for having there a very opinionated guide on how to develop in Julia, before just putting people to play with the REPL.

[ChrisRackauckas]

Actionable: Higher level error message on broadcast `materialize!` failures · Issue #45086 · JuliaLang/julia · GitHub

[isaacsas]

Seems like there is an ambiguity here because of there being both in and out of place methods, and SciMLBase.jl/utils.jl at 38c31b6fdea140e54697e25cb46bb3ad9eeb65c7 · SciML/SciMLBase.jl · GitHub auto-selects the version with the most arguments when there are multiple methods. Couldn’t u0 also be looked at to determine which version to select when there are multiple methods defined?

[ToucheSir]

@jakobnissen

Yes, but AFAIK, Documenter’s autogenerated documentation is not very thorough. It adds the docstring. It doesn’t for example, add links to functions automatically found with methodswith , or tell which position in the type hierarchy a type is, or which type parameters a type has.

Julia has pretty good human-written docs, but I think Rust shows the autogenerated docs could be better.

This is the gripe I have with the status quo around documentation generation. You may be interested in GitHub - lorenzoh/Pollen.jl: Format-independent document generation tool built for interactive work. It’s not at rustdoc levels of automation, but IMO it’s taking an important step towards that and it is already deployed in FluxML (with hopefully more repos to follow).