Square plots in GR

I am currently trying to make a contour plot of a a 100x100 matrix using Plots with the GR backend. Since it is a square matrix I would like the contour plot to be square also. Someone suggested to me using aspect_ratio=:equal to achieve this, but rather than adding whitespace around the figure, it adds it inside the axis, so the plot looks odd, as can be seen below.

My end goal would be a layout similar to this

For a square-domain, you can use a single keyword lims (otherwise: xlims, ylims)

Thank you so much, this worked perfectly.