You need to call repaint() and revalidate(). The former tells Swing that an area of the window is dirty (which is necessary to erase the image of the old children removed by removeAll()); the latter tells the layout manager to recalculate the layout (which is necessary when adding components). This should cause children of the panel to repaint, but may not cause the panel itself to do so (see this for the list of repaint triggers).
On a more general note: rather than reusing the original panel, I’d recommend building a new panel and swapping them at the parent.
