I have always strived to adhere strictly to this rule: never use a technique or method that you don’t understand. So I generally use packages just like everyone else, but what I like so much about Julia is that I think it’s the best language for learning new things. The fact that most library code is written in Julia is a big part of that, but just the ergonomics and syntax of the language make it great for learning and implementing new algorithms and methods yourself. I have been lucky enough to be able to take the time to implement a lot of algorithms myself, but it’s usually just an academic exercise to make sure I understand exactly how it works and what it’s doing. Once I feel that I have a good grasp of the technique, I’m more than happy to rely on a good package!
Just as an example, I asked Claude to put together an elementary gradient boosting algorithm, and it gave me:
using Statistics
# Best single split on x, minimizing squared error of residuals r
function fit_stump(x, r)
best = (sse = Inf, t = 0.0, left = 0.0, right = 0.0)
for t in unique(x)
L, R = r[x .<= t], r[x .> t]
(isempty(L) || isempty(R)) && continue
sse = sum(abs2, L .- mean(L)) + sum(abs2, R .- mean(R))
sse < best.sse && (best = (; sse, t, left = mean(L), right = mean(R)))
end
return best
end
predict(s, x) = ifelse.(x .<= s.t, s.left, s.right)
# Squared loss: the negative gradient is just the residual
function boost(x, y; rounds = 100, η = 0.1)
f0 = mean(y)
F = fill(f0, length(y))
stumps = []
for _ in 1:rounds
s = fit_stump(x, y .- F)
F .+= η .* predict(s, x)
push!(stumps, s)
end
return xnew -> f0 .+ η .* sum(predict(s, xnew) for s in stumps)
end
x = range(0, 2π, length = 200)
y = sin.(x) .+ 0.1 .* randn(200)
model = boost(x, y)
mean(abs2, y .- model(x)) # training MSE, close to the noise level
If I was studying this, I’d normally fire up a Pluto notebook, add in some sliders, etc., and spend a couple of hours looking at how the pieces fit together, how changing the inputs changes the outputs, etc. Then I might go over to EvoTrees.jl and browse their code for another couple of hours until I felt confident I could explain to you (at least to a reasonable degree) how the algorithm works. Only then would I actually use it in practice.
I think it’s fair to say that if you wanted to adopt this approach in Python or R, it’s a lot more difficult and you would frequently have to roll up your sleeves and dig into some C++ to get to the interesting parts of the package code…