Set Matplotlib colorbar size to match graph

I cannot get the colorbar on imshow graphs like this one to be the same height as the graph, short of using Photoshop after the fact. How do I get the heights to match?