Found it! An Axis has a field called attributes, in which I found xlabelsize => 16.0. This can be set either by
plot(xvalues, yvalues, axis=(xlabelsize=20))
or by having an Axis object ax, and setting the value by ax.xlabelsize=20.
On doing it globally, it is explained in Theming · Makie Plotting Ecosystem (juliaplots.org). MWE, copied from the documentation:
fontsize_theme = Theme(fontsize = 20)
set_theme!(fontsize_theme)