jQuery removeClass wildcard

Is there any easy way to remove all classes matching, for example,

color-*

so if I have an element:

<div id="hello" class="color-red color-brown foo bar"></div>

after removing, it would be

<div id="hello" class="foo bar"></div>

Thanks!