For finer control of the spacing, we can first plot an empty line label with plots() keyword argument xlabel="\n" and then annotate outside the plot using the plot units:
xlabel="\n"
annotate!([π/2],[-0.2], text("X axis", 11, "Computer Modern"))
to get:
