Ah, my mistake was using pt_per_unit in the wrong place… it should be used when saving the PDF. So it was using the default value (0.75). I can also use one of the default themes instead of specifying the font myself. Here is the solution that works.
using CairoMakie
x = 0:0.01:2π
y = cos.(x);
pt_per_unit = 1
figsizeinches(width, height) = (width, height) .* (72/pt_per_unit)
height = 2.5 # inches
width = 4.5 # inches
fontsize = 10
set_theme!(theme_latexfonts(); fontsize)
f = Figure(; size=figsizeinches(width, height))
ax = Axis(f[1,1], xlabel="x label", ylabel="y label")
lines!(ax, x, y, label="cosine", color=:tomato)
axislegend(position = :rb)
save("cosine.pdf", f; pt_per_unit)
As @p_f mentioned, maybe a better way than inches is basing the size of the figure on the \textwidth of the LaTeX document, and using pt_per_unit=1