You can also try this (seems to work well also in IJulia)
PyPlot.matplotlib[:rc]("mathtext",fontset="cm") #computer modern font
PyPlot.matplotlib[:rc]("font",family="serif",size=12) #font similar to LaTeX
figure(figsize=(7,5))
plot(x,y)
title(L"a title using LaTeX, $2 b^2 + 0.5$")
xlabel(L"$x$")
ylabel("function value")
text(-2.5,0.9,L"some text, $\ln(\mathrm{loss})$")
text(-2.5,5,L"$\mu_2 = \int x^2 f(x) dx$")
In case you like TImes-Roman, try this
PyPlot.matplotlib[:rc]("mathtext",fontset="stix")
PyPlot.matplotlib[:rc]("font",family="STIXGeneral",style="normal",size=18)
...