What does “Could not find or load main class” mean?

A common problem that new Java developers experience is that their programs fail to run with the error message: Could not find or load main class ...

What does this mean, what causes it, and how should you fix it?