One thing that is definitely suboptimal is sleeping for 1/fps, because that only gives you a framerate of fps if you do exactly nothing in between. In any other case, you drop frames. The simplest way to avoid this is to use a non-drifting update loop, where you sleep only the difference between the elapsed time and the necessary inter-frame interval.
I just tested it a bit as well, and on my macbook the 200x200 px heatmap took around 30ms to update, so that would fit your observation. That is not great, but if you then also sleep 33ms you halve your framerate unnecessarily