Welcome to the Julia community!
It can be done by overriding this function like so:
using Compose, Gadfly
function Gadfly.display(d::Gadfly.GadflyDisplay, p::Union{Plot,Compose.Context})
display(d, Gadfly.default_mime(), p)
return
end
Then a test:
set_default_plot_format(:svg)
plot(x=1:10, y=rand(10), Geom.point)
import Cairo
set_default_plot_format(:pdf)
plot(x=1:10, y=rand(10), Geom.point)
The plot will open in whatever your default program for svg, pdf is.